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ABSTRACT 


The  pertinent  points  of  visual  neurophysiology  and 
neuroanatcmy  are  reviewed  with  particular  enphasis  on 
how  retinal  light  distributions  result  in  perceived 
phenoaena.  The  neural  modeling  techniques  used  at  the 
Naval  Postgraduate  School  are  discussed.  The  specific 
coaputer  programs  used  by  the  author  in  modeling  are 
described.  These  stem  from  a basic  model  capable  of 
calculating  the  postsynaptic  potential  and  spike 
outputs  for  any  sequence  of  excitatory  and  inhibitory 
inputs.  More  advanced  programs  model  facilitation, 
fatigue,  and  narrow  band  motion  detection.  The  most 
advanced  program  models  lateral  inhibition  in  an  eight 
neuron  linear  array.  The  lateral  inhibition  network  is 
used  to  show  temporal  phenomena  (null  vs.  preferred 
direction,  fast  vs.  slow  speed  detection)  as  well  as 
spatial  phenomena  (Bach  bands,  line  sharpening, 
disinhibition,  spatial  frequency  response)  . Bany 
suggestions  for  future  modeling  work  are  given. 
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I.  INTBOD  DCTION 


The  visual  system  is  a truly  amazing  thing.  It  must,  as 
is  true  of  anything  having  to  do  with  living  organisms,  be 
fully  describable  both  anatomically  and  functionally  in 
terms  of  well  defined  physical  principles.  This  is  certainly 
true  at  present  of  the  optical  end  of  the  visual  system, 
from  the  cornea  to  the  retina.  There  is  very  little  mystery 
to  the  refraction  , reflection,  and  diffraction  of  light  in 
the  structures  of  the  eyeball.  But  there  is  still  a great 
deal  of  mystery  associated  with  the  nervous  system  end  of 
the  visual  system,  even  though  much  hard,  reproducible 
scientific  data  exists. 

These  data  tend  to  be  of  four  general  types.  In 
animals,  data  consists  largely  of  microelectrode  studies  of 
cellular  electrical  activity  in  response  to  various  visual 
or  electrical  stimuli.  In  humans  it  is  generally  perceptual 
in  nature  , the  subject  being  presented  with  various  visual 
stimuli  at  or  near  the  threshold  of  detection  and  being 
asked  for  a verbal  response  upon  detecting  the  stimulus.  One 
exception  is  the  electroencephalogram,  recorded  using  scalp 
electrodes  as  the  subject  is  presented  with  various  stimuli. 
Of  course,  there  are  the  all-important  histological  studies 
of  both  human  and  animal  nervous  tissue,  whose  aim,  in 
general,  is  to  ascertain  functional  anatomical  features, 
such  as  interconnections  between  neurons.  All  four  of  these 
data  types  are  of  great  value  in  aiming  toward  a solution  of 
how  visual  systems  work.  But  what  has  been  conspicuously 
missing  until  recently  are  suggestions  and  speculations  as 
to  what  circuits  of  neural  elements  could  be  responsible  for 
visual  phenomena.  It  is  the  aim  of  this  thesis  to  review 


what  is  known  about  the  neurophysiology  of  visual  systems, 
and  then  tc  propose  some  models  of  neurons  and  neural 
circuits  which  could  explain  some  of  the  observed  perceptual 
phenomena.  It  must  be  restated  for  emphasis  that  the  results 
of  the  modeling  in  this  thesis,  although  based  upon  the  best 
information  available,  are  necessarily  highly  speculative, 
and  should  be  considered  as  how  certain  phenomena  might 
occur  rather  than  how  they  do  occur. 
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The  process  we  call  vision  begins  when  photons  from  a 
scene  enter  the  cornea  and  ends  when  the  visual  association 
area  of  the  brain  conveys  meaning  to  the  organism.  It  is 
obvious  that  a vast  amount  of  information  processing  occurs 
in  order  for  this  transition  to  occur.  Much  information 
which  enters  the  cornea  is  lost  to  the  organism,  but  this  is 
an  adaptive  advantage  rather  than  a disadvantage.  The 
organism  must  respond  quickly  to  the  important  features  of 
the  visual  field.  If  all  incident  information  were  reguired 
to  be  processed  and  presented  to  the  nervous  system  of  the 
organism,  the  response  could  not  be  as  rapid.  It  is 
apparent,  therefore,  that  visual  systems  must  have  means  for 
extracting  the  important  aspects  of  a scene.  The  aspects 
which  are  considered  important  vary  from  species  to  species, 
but  two  things  tend  to  be  extracted  by  the  visual  systems  of 
all  organisms  studied:  high  contrast  features  and  movement. 


A.  THE  NEUBGN  AND  THE  SYNAPSE 


Before  discussing  the  processing  of  information  by  the 
visual  system,  the  basic  functional  units  of  which  all 
nervous  tissue  is  built  will  be  described. 


The  neuron,  or  nerve  cell,  consists  of  a cell  body 
(soma),  an  input  end  (dendrites),  and  an  output  end  (axon), 
as  shown  in  Fig  1.  The  point  at  which  two  nerve  cells 
communicate  is  called  a synapse.  Consider  a nerve  impulse 
(action  potential)  proceeding  to  the  right  at  point  A.  Upon 


reaching  the  dendrites  of  neuron  1,  the  propagating  action 
potential  causes  soae  subaicroscopic  change  which  causes  the 
neurotransaitter  vessicles  to  duap  their  contents  into  the 
synaptic  cleft.  Hitochondria  are  present  in  the  bouton  to 
provide  the  energy  necessary  for  this  release.  After 
diffusing  across  the  cleft,  the  neurotransaitter  causes  ion 
peraeability  changes  in  the  subsynaptic  aeahrane  which  in 
turn  cause  aeabrane  potential  changes. 

The  aeabrane  would  have  a resting  potential  of  -80 
aillivolts  (inside  negative  with  respect  to  the  outside). 
The  effect  of  an  excitatory  neurotransaitter  would  be  to 
depolarize  the  aeabrane. 

By  a process  called  electrotonic  spread,  the  potential 
change  at  the  dendrites  is  transaitted  by  ion  currents  to 
the  axon  hillock.  If  the  aeabrane  potential  at  the  axon 
hillock  reaches  roughly  -60  aillivolts,  a draaatic  change  in 
aeabrane  peraeability  to  ions  causes  a propagating  polarity 
reversal  (the  action  potential) , which  travels  the  length  of 
the  axon  cf  neuron  four. 

If  other  E-inputs  are  occurring,  say  at  neuron  two, 
spatial  suaaation  would  occur.  This  neans  that  inputs  at 
different  dendritic  locations  will  each  cause  a change  at 
the  axon  hillock,  and  can  work  together,  or  sua,  to  produce 
an  action  potential. 

If  a tine  sequence  of  E-inputs  occurs  at  one  dendrite, 
teaporal  suaaation  occurs.  This  aeans  that  inputs  at 
different  tines  can  work  together,  or  sua,  to  produce  an 
action  potential,  so  long  as  the  inputs  are  not  too  far 
apart  in  tiae.  Figure  1 shows  a typical  post  synaptic 
response  (FSB),  which  is  the  change  in  aeabrane  potential  at 
the  axon  hillock  due  to  a single  input  at  a dendrite.  The 
finite  duration  of  the  PSB  aeans  that  inputs  occurring  in 
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rapid  succession  will  not  each  have  to  start  anew  froa  rest 
potential,  but  will  have  soae  potential  change  left  over 
froa  previous  inputs. 

Inhibitory  as  well  as  excitatory  neurons  exist.  If  a 
neuron  is  inhibitory,  it  releases  a neurotransaitter  which 
causes  postsynaptic  hyperpolarization  vice  depolarization. 
That  is,  it  tends  to  inhibit  downstreaa  neurons  froa 
producing  outputs.  The  shape  of  the  inhibitory  PSB  is  an 
inverted  version  of  the  excitatory  PSB,  and  suaaation  at  the 
axon  hillock  is  a subtraction.  A neuron  is  either  wholly 
excitatory  or  wholly  inhibitory — never  mixed. 

B.  THE  VISUAL  PATHWAYS 

Pigure  2 depicts  the  visual  processing  pathways.  Light 
enters  the  eyes  and  is  focused  on  the  retina  by  the  cornea 
and  lens  structures.  As  quanta  of  light  are  absorbed,  the 
retinal  receptors  respond  by  varying  aeabrane  potentials. 
These  varying  potentials  are  coupled  through  the  cells  of 
the  retina  to  the  ganglion  cells,  whose  fibers  fora  the 
optic  nerve.  The  ganglion  cells  code  the  visual  scene  into 
spike  action  potentials  for  transmission  to  the  brain.  The 
inforaation  leaving  the  retina  has  been  processed  by  the 
retinal  neural  circuits,  and  therefore  is  not  siaply  a coded 
version  of  the  scene. 

At  the  optic  chiasa,  fibers  undergo  a rerouting  such 
that  all  inforaation  coaing  froa  the  right  half  of  the 
visual  scene  goes  to  the  left  half  of  the  brain,  while 
inforaation  ccaing  froa  the  left  half  of  the  visual  scene 
goes  to  the  right  half  of  the  brain. 

At  the  lateral  geniculate  nuclei,  the  fibers  synapse 
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with  other  neurons.  The  neural  interconnections  in  the 
lateral  geniculate  nuclei  process  the  visual  scene  even 
■ore.  It  is  believed  that  the  interleaving  of  inforaation  in 
the  lateral  geniculate  nuclei  fron  the  sane  scene  portion 
but  froa  different  eyes  constitutes  processing  which 
ultiaately  results  in  depth  perception. 

Fibers  free  the  neurons  of  the  lateral  geniculate  nuclei 
terainate  cn  neurons  of  the  visual  cortex.  Here,  the 
processing  has  proceeded  to  the  degree  that  specific  visual 
features  at  specific  locations  are  being  extracted.  These 
specific  features  of  the  visual  scene  are  pieced  together  in 
the  visual  association  areas,  and  ultiaately  result  in  the 
organisa's  being  able  to  interact  with  his  visual 
environaent. 

Mot  shown  is  the  pons  of  the  brain,  located  i£the  brain 
stea.  The  pons  also  receives  visual  inforaation  and  is  an 
iaportant  center  for  processing  this  inforaation. 

C.  THE  TARGET  NEURON  CONCEPT 

The  presence  of  a particular  feature  is  nearly  always 
signalled  by  strong  spike  outputs  froa  a single  neuron.  This 
neuron  is  usually  retinal  (a  ganglion  cell) , geniculate, 
cortical,  or  pontine,  and  is  teraed  a target  neuron  in  this 
thesis,  since  auch  inforaation  is  targeted  at  or  converged 
to  the  neuron.  The  output  tends  to'  be  binary  in 
nature^-either  the  condition  is  net,  in  which  case  frequent 
outputs  occur,  or  the  condition  is  not  net,  in  which  case 
few  or  no  outputs  occur.  In  general,  the  siaplest  sets  of 
criteria  for  output  production  occur  early  in  the  visual 
pathway,  while  very  coaplicated  sets  of  criteria  generally 
apply  later. 
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An  example  of  a staple  criterion  is  coamon  for 
ganglion  and  geniculate  cells.  Teraed  the  "on-center"  cells, 
strong  outputs  occur  when  receptors  feeding  then  are 
illuminated  by  a saall  dot  of  light.  The  dual  of  this 
situation  occurs  for  the  "off-center"  cells — the  saall  dot 
of  light  causes  the  ganglion  cell  to  have  no  outputs, 
whereas  in  tctal  darkness,  the  cell  would  have  outputs. 

In  addition  to  simply  turning  on  or  off  in  response 
to  a saall  dot  of  light,  ganglion  cells  respond  oppositely 
to  bright  surround  stimuli.  That  is,  when  an  annulus  of 
light  illuainates  receptors  surrounding  the  receptors  which 
feed  a ganglion  cell  of  the  on-center  type,  the  result  is  an 
off  response.  Similarly,  a surround  stimulus  for  an 
off-center  ganglion  cell  produces  an  on  response.  The  cause 
of  the  surround  stimulus  effects  has  been  shown  to  be 
inhibitory  interneurons  (horizontal  and  amacrine  cells) 
which  spread  their  inhibition  laterally. 

2.  saais niils 


Geniculate  cells  remain  fairly  simple  in  their 
properties.  They  are  very  similar  to  ganglion  cells,  but 
have  an  enhanced  ability  for  surround  stimuli  to  counteract 
center  stimuli.  Thus  they  appear  to  represent  a higher 
degree  of  capability  for  extracting  information  concerning 
contrast. 

3. 
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Examples  of  nore  involved  sets  of  criteria  occur 
cortically,  where  "simple"  as  well  as  "complex"  cells  have 
been  established  £ Bef  10].  Simple  cells  respond  most 
favorably  to  line  stimuli  which  have  the  correct  position  in 
the  visual  field  and  the  correct  angular  orientation. 
Complex  cells  respond  strongly  for  line  stimuli  with  proper 
orientation  and  direction  of  movement,  and  tend  to  have 
larger  "on"  fields.  Complex  cells  are  believed  to  be  targets 
for  outputs  of  many  simple  cells  which  detect  the  same 
orientation.  In  some  way  (possibly  unilateral  spread  of 
inhibition)  the  neural  interconnections  mediate  the  motion 
direction  specificity  of  the  complex  cells.  Thus  the  simple 
and  complex  cortical  cells  present  signals  tc  the 
association  areas,  each  signal  having  its  cwn  meaning  in 
terms  of  the  content  of  the  visual  field.  Signals  from  many 
millions  of  these  cells  interconnect  with  many  millions  of 
association  cells  to  provide  the  organism  with  a conscious 
perception  of  the  scene. 

* • SSfitiEe 


The  pons  of  the  brain  is  also  an  important  target 
area  for  visual  information.  It  is  believed  that  the  pons 
is  important  in  assembling  visual  information  from  the 
visual  cortex  and  relaying  this  information  to  the 
cerebellum  for  use  in  motor  control  £Ref  6].  A typical 
pontine  cell  responds  strongly  to  motion  in  one  direction 
only  and  is  only  weakly  stimulated  or  even  inhibited  by 
motion  in  the  opposite  direction.  Small  stimuli  are  most 
effective  in  producing  strong  outputs;  shape  and  orientation 
are  not  generally  important.  An  individual  target  cell  would 
respond  most  strongly  to  motion  at  one  particular  angular 
velocity.  The  total  range  of  angular  velocities  covered  by 
all  cells  recorded  was  roughly  ten  to  800  degrees  per  second 
[ Bef  8]. 
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D.  THE  RETINA 


Because  the  response  of  retinal  ganglion  cells  is  not  a 
siaple  function  of  photons  falling  on  connected  receptors, 
it  is  apparent  that  retinal  processing  does  indeed  gc  on. 
The  retina  is  important  enough  to  deserve  a description 
here,  however  brief  and  oversimplified,  because  it  contains 
two  systems  of  laterally  inhibiting  interneurons.  Lateral 
inhibition  is  very  fundamental  to  the  functioning  nervous 
systea. 

The  retina  is  pictured  in  Fig  3 in  cross  section.  Light 
illuainates  the  receptor  cells  after  passing  through  the 
eyeball  and  the  other  neural  layers.  Photon  absorptions 
cause  changes  in  visual  pigaent  substances  which  in  turn 
cause  variations  in  the  aeabrane  potential  of  the  receptor 
cells.  Eecent  evidence  indicates  that  this  aeabrane 
potential  variation  is  coupled  to  the  bipolar  and  horizontal 
cells  by  aodulating  the  flow  of  an  unidentified  chemical 
neurotransaitter  substance.  The  bipolar  and  horizontal  cells 
are  very  saall  in  length  as  coapared  with  normal  "long-axon" 
neurons,  and  therefore  do  not  need  to  eaploy  spikes  to 
transait  information  along  their  length.  The  inforaation  is 
in  the  fora  cf  subthreshold  aeabrane  potential  changes  (slow 
potentials)  which  travel  electrotonically  and  aodulate  the 
release  of  a neurotransaitter  just  as  did  receptor  potential 
variations.  Bipolar  cells  terainate  on  aaacrine  and  ganglion 
cells  (in  priaates) . Aaacrine  cells  are  larger,  depolarize 
when  stiaulated  (receptors,  horizontal,  and  bipolar  cells 
hyperpolarize) , and  tend  to  have  a propensity  for  spike 
production.  Horizontal  and  aaacrine  cells  are  inhibitory, 
while  the  other  retinal  cells  are  excitatory.  The  axons  of 
ganglion  cells  fora  the  optic  nerve,  in  which  all 


information  is  in  the  fora  of  action  potentials.  Because  no 
other  connection  exists  between  the  retina  and  the  brain, 
these  action  potentials  constitute  the  total  result  of 
retinal  processing  of  the  visual  scene. 

Several  inportant  points  need  to  be  nade  about  the 
neural  interconnections  just  discussed.  Each  ganglion  cell 
has  associated  with  it  a receptive  field.  Foveally,  there 
are  few  receptors  feeding  each  ganglion  cell.  These 
receptors  are  exclusively  cones,  and  fairly  high  levels  of 
light  are  reguired  for  vision.  Peripherally,  as  many  as  a 
hundred  receptors,  mostly  rods,  feed  each  ganglion  cell, 
thus  enhancing  sensitivity  at  the  expense  of  acuity.  The 
most  important  point,  however,  is  that  the  information 
leaving  a certain  ganglion  cell  depends  not  merely  upcn  the 
brightness  falling  upon  the  receptors  feeding  it,  but  also 
depends  upon  what  light  distribution  falls  upon  adjacent 
areas.  The  horizontal  and  amacrine  cells  perform  this 
function  by  extending  laterally,  and  by  inhibiting  the 
neurons  upon  which  they  terminate.  This  explains  the  term 
"lateral  inhibition,"  and  also  explains  why  ganglion  cells 
respond  oppositely  to  bright  surround  stimuli. 

Another  important  type  of  processing  which  occurs 
retinally  is  the  development  of  combined  color  signals.  Each 
of  three  types  of  cone  cells  has  its  own  response  spectrum, 
with  significant  overlap.  Ultimately,  the  signals  from  the 
three  receptor  types  must  be  compared  to  give  the  organism 
the  proper  color  sensation.  There  is  evidence  that  this 
comparison  begins  retinally  by  the  formation  of  difference 
signals  at  the  ganglion  cell  level.  Although  color  visual 
processing  will  not  be  modeled  in  this  thesis,  this  was 
mentioned  to  point  out  that  other  types  of  processing  do  go 
on  retinally.  Additionally,  this  is  another  example  cf  how 
visual  processing  occurs  in  stages,  with  early  development 
occurring  retinally,  and  final  features  being  extracted 
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cortically 


Thus  the  retina  is  capable  of  coding  the  visual  scene 
into  action  potentials  which  go  to  the  brain.  The  coding  is 
the  result  of  the  total  spatial  distribution  of  light  on  the 
retina — not  aerely  on  each  local  point  of  light  taken 
independently.  This  lays  the  ground  work  for  modeling  of 
spatial  pbenoaena  later  in  this  thesis. 

The  respcnse  of  a ganglion  cell  can  reflect  temporal 
properties  as  well  as  spatial  distributions  of  light.  In  the 
audpuppy  (a  saall  fish) , aicroelectrode  measurements  have 
confirmed  that  one  type  of  ganglion  cell  responds  to 
stationary  light  distributions  while  a different  type  of 
ganglion  cell  responds  only  to  changes  in  light 

l 

distributions.  This  means  that  retinal  processing  has 
occurred  which  has  extracted  motion  information  from  the 
visual  field.  In  the  frog,  there  are  five  types  of  ganglion 
cells,  one  of  which  is  very  specific  for  a saall,  convex, 
dark  object  (such  as  a fly)  moving  through  the  receptive 
field  of  that  cell.  This  represents  a very  high  degree  of 
motion  detection  capability  located  ri-jht  in  the  frog's 
retina.  In  other  vertebrates,  similar  target  cells  exist, 
but  at  higher  centers  such  as  the  pons,  the  superior 
colliculus,  and  the  visual  cortex. 

The  point  of  discussing  retinal  motion  detectors  is  to 
point  out  that  since  the  "equipment"  and  types  of  neural 
interconnections  available  retinally  are  very  well  known, 
the  number  of  ways  in  which  this  system  can  work  are 
limited.  A lateral  inhibition  network  can  be  used  to  model 
a motion  detector.  Furthermore,  the  presence  of  such  a 
retinal  notion  detecting  system  in  the  frog  lends  credence 
to  any  such  model. 
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E,  SPATIAL  C0NSIDE3ATI0NS 


1 • Sharpening 


A neural  network  with  lateral  inhibition  can  sharpen 
details  in  a visual  scene.  The  entry  of  light  into  the 
optical  end  of  the  eyeball  is  accompanied  by  scattering  and 
diffraction,  both  of  which  tend  to  create  a blurred  image  on 
the  retina  cf  an  object  which  should  be  perceived  as  having 
sharp  contrasts.  The  end  result  of  this  process  is  that 
cortical  cells  responsible  for  detecting  a sharp  line,  edge, 
or  slit  at  a certain  angular  orientation  at  a certain  point 
in  the  visual  field  can  do  so  more  efficiently.  This  process 
will  be  explained  more  fully  later,  as  models  are  presented. 


2 . Spatial  Frequency 


A useful  concept  in  dealing  with  visual  scenes  is 
spatial  freguency.  This  term  refers  to  the  frequency,  in 
cycles  per  degree  of  visual  angle,  at  which  luminosity 
varies  from  light  to  dark.  The  simplest  case  is  a level  of 
luminosity  which  does  not  change  with  position  (zero  cycles 
per  degree) . The  next  most  fundamental  example  is  a field 
where  there  is  no  variation  vertically,  but  horizontally  the 
luminosity  varies  from  light  to  dark  as  the  sine  of 
position.  This  is  called  a sinusoidal  grating.  Another 
example  is  a series  of  vertical  light  and  dark  bars,  the 
so-called  square  wave  grating.  Just  as  with  any  periodic 
time  series  waveform,  these  periodic  spatial  gratings  can  be 
considered  as  being  the  sun  of  all  Fourier  components,  each 
component  being  a pure  sine  at  some  harmonic  of  the 
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fundamental  frequency 


3 . The  Spatial  Modulation  Transfer  Function 

The  visual  system  responds  differently  to  different 
spatial  frequencies.  In  perceptual  studies,  the  subject  is 
asked  to  signal  when  he  just  detects  a sinusoidal  grating 
flashed  on  a screen.  Each  grating  flashed  has  two  basic 
properties:  spatial  frequency  and  contrast,  contrast  refers 
to  the  difference  in  luminosity  between  the  lightest  and 
darkest  regions.  Hhen  the  data  is  plotted,  it  is  found  that 
roughly  three  cycles  per  degree  is  the  spatial  frequency  at 
which  subjects  can  detect  gratings  with  the  lowest  contrast. 
As  spatial  frequency  gets  higher  or  lower  than  three  cycles 
per  degree,  mere  contrast  is  required  for  detection.  The 
resulting  curve  of  spatial  frequency  versus  contrast 
threshold  for  detection  is  called  the  spatial  modulation 
transfer  function.  It  is  considered  as  the  best  estimate  of 
the  spatial  frequency  response  of  the  human  visual  system. 

The  high  frequency  fall-off  of  the  spatial 
modulation  transfer  function  is  due  in  part  to  the  optical 
properties  of  the  cornea,  lens,  and  humor  of  the  eye.  Any 
lens  system  has  an  upper  limit  to  its  resolution,  and 
transfer  functions  have  long  been  used  to  describe  the  high 
spatial  frequency  performance  of  lens  systems.  The  finite 
size  of  the  individual  receptors,  the  scattering  of  light  as 
it  impinges  upon  the  retina,  and  the  summation  from 
receptors  to  bipolar  cells  are  all  believed  to  play  roles  in 
limiting  the  high  frequency  response. 

The  low  frequency  fall-off  is  much  harder  to 
comprehend,  and  involves  a comparison  of  the  visual  angle 
subtended  by  one  cycle  of  pattern  with  the  visual  angle 
through  which  inhibition  from  a pinpoint  of  light  would 
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spread.  Hben  inhibition  spreads  through  one-half  cycle, 
outputs  frci  ganglion  cells  are  at  enhanced  contrast  levels. 
This  is  because  strong  inhibition  froa  the  bright  area 
inhibits  the  dark  area,  while  lack  of  inhibition  froa  the 
dark  area  allows  maximum  output  fron  the  bright  area,  thus 
enhancing  contrast.  It  is  iaportant  to  note  here  that  since 
inhibition  does  not  begin  until  several  neurons  are 
traversed  radially,  the  centers  of  the  bright  peaks  dc  not 
inhibit  the  edges  of  the  bright  regions.  Next,  consider  a 
situation  in  which  the  lateral  extent  of  inhibitory  spread 
is  auch  less  than  one  cycle  of  pattern.  The  result  here 
would  be  that  bright  areas  inhibit  themselves  while  dark 
areas  receive  little  inhibition  from  themselves.  The  effect 
would  be  a less  of  contrast. 

The  low  response  at  low  frequencies  as  well  as  the 
contrast  enhancing  response  at  high  frequencies  will  be 
BOdeled  in  this  thesis. 
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III.  THE  MODELS 


A.  THE  COMPOTES  FACILITY 

The  computer  modeling  for  this  thesis  was  all  performed 
using  the  PDP-11/40  digital  computer,  made  by  Digital 
Eguipment  Corporation  of  Maynard,  Mass.,  and  located  in  the 
Bioengineering  Laboratory  at  the  Naval  Postgraduate  School. 
This  computer  is  eguipped  with  32K  words  (1K=1024,  one  word 
eguals  16  bits)  of  readily  addressable  core  memory,  two 
RK-11  disk  drives,  a TC-11  magnetic  tape  drive  pair,  and  an 
LA-30  keyboard/character  printer.  Block  data  is  displayed 
on  a storage  cathode  ray  tube  and  plotted  by  a Hewlett 
Packard  7004B  X-Y  recorder. 

All  neural  modeling  programs  in  existence  when  inherited 
by  the  author  were  written  in  Time  Series  Language  (TSL) 
with  links  to  machine  language  routines  where  necessary. 
TSL  is  a higher  level  language  especially  designed  for 
manipulating  blocks  of  data.  There  are  advantages  as  well  as 
disadvantages  to  using  a higher  level  language  in  general 
and  TSL  specifically.  On  the  positive  side,  programs  are 
easy  to  change,  as  moving  of  statements  is  done 
automatically.  Also,  the  programmer  need  know  nothing  about 
the  processor  and  memory  utilization  in  order  to  perform 
certain  limited  programs.  But  in  order  to  extract  the 
fullest  performance  from  the  32K  words  of  core,  speed 
program  execution  somewhat,  know  exactly  where  in  core 
stacks  and  blocks  are  located,  and  generally  know  what  is 
happening,  there  is  no  substitute  for  complete  machine 


language  programing.  To  this  end.  Dr.  Uarnont  has 
painstakingly  developed  the  APTEC  "language,"  which  is  a 
collection  cf  frequently  used  data  processing  and  servicing 
routines.  All  neural  modeling  programs  used  in  this  thesis 
use  APTEC  and  pure  machine  language. 

B.  THE  BASIC  NEUROH  MODEL 

The  object  of  the  basic  excitatory  and  inhibitory 
postsynaptic  potential  (EIPSP)  program  is  to  form  the  proper 
PSP  response  given  a scenario  of  excitatory  and  inhibitory 
inputs  at  a neuron's  dendrites  (Fig  4a).  The  accomplishment 
of  this  rather  modest  sounding  task  took  considerable  tine, 
talent,  and  guidance,  and  was  accomplished  by  the  students 
of  Dr.  Marnont  during  1975  as  work  for  his  course  in 
computer  modeling.  The  operation  of  the  EIPSP  program  will 
be  reviewed  here,  since  it  forms  the  basis  upon  which  later 
more  elaborate  modeling  programs  were  built. 

First,  the  properly  shaped  responses  for  excitatory  as 
well  as  inhibitory  inputs  are  formed  in  a pair  of  IK  blocks. 
These  are  termed  the  excitatory  and  inhibitory  postsynaptic 
responses  (EPSR  and  IPSR)  and  are  shown  in  Fig  10.  Then  a IK 
PSP  processing  block  is  established  along  with  a pointing 
register  for  use  in  forming  the  PSP  at  each  word  of  the 
block.  Also  needed  are  excitatory  and  inhibitory  input 
counting  registers,  and  two  stacks  for  keeping  track  of  the 
proper  age  of  each  input  which  has  occurred  since  the  last 
output. 

Processing  begins  by  testing  for  the  presence  of  an 
excitatory  input  at  the  first  word  of  the  block  (Fig  5) . If 
an  input  is  present,  a zero  age  marker  is  placed  on  the 
E-age  stack,  and  the  E-counter  is  incremented.  The  E-ccunter 
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is  then  tested  to  determine  whether  any  inputs  requiring  PSP 
update  have  occurred  since  the  last  output.  If  the  counter 
is  positive  a nifty  naneuver  forming  the  heart  of  EIPSP  is 
accoaplishe d.  Using  the  E-input  counter  as  a loop  counter 
and  the  E-age  stack  as  the  source  of  ages  for  each  E-input 
which  has  occurred  since  the  last  output,  the  PSP  word  is 
formed  by  adding  together  the  proper  nuaber  of  properly  aged 
EPSfi's.  The  E-age  stack  is  then  aged  by  one  word  to  prepare 
it  for  the  next  processing  pass. 

This  identical  procedure  is  followed  for  1-inputs, 
except  that  the  aged  IPSB's  are  subtracted  instead  of  added 
when  foraing  the  PSP  word. 

Hext  the  resultant  PSP  word  is  compared  with  the 
threshold  for  spike  initiation.  If  below  threshold, 
processing  proceeds  to  the  next  word  of  the  block.  If  equal 
to  or  greater  than  threshold,  a marker  is  placed  in  the 
output  block,  the  E and  I-age  stacks  are  reset,  the  E and 
I-input  counters  are  zeroed,  and  the  processing  block 
pointer  is  advanced  to  the  end  of  the  refractory  period.  A 
test  is  then  made  to  deteraine  whether  or  not  the  pointer  is 
at  the  end  of  the  processing  block.  If  so,  processing  ends 
and  the  results  are  displayed.  If  not,  the  pointer  is 
advanced  to  the  next  word  of  the  block  and  the  entire 
seguence  repeated. 

Mo  effort  was  made  to  aodel  the  aeabrane  action 
potential  which  exists  during  output.  Rather,  no  processing 
whatever  is  done  during  this  tiae,  leaving  the  PSP  egual  to 
zero.  (The  PSP  processing  block  was  set  to  zero  prior  to 
beginning  processing) . The  action  potential,  if  aodeled, 
would  be  strictly  cosmetic,  since  no  use  would  be  made  of 
the  aeabrane  potential  excursion  which  would  occur. 

This  basic  neural  aodel  is  extremely  useful  in  building 


30 


functional  neural  circuits.  This  is  done  in  its  simplest 
form  by  storing  the  IK  output  Barker  block  on  disk  and 
employing  that  as  an  excitatory  or  inhibitory  input  to 
subsequent  neurons,  thus  aodeling  divergence.  Similarly, 
more  than  one  output  block  can  be  added  together  and  applied 
as  the  E or  I-input  to  a subsequent  neuron  or  neurons,  thus 
modeling  convergence.  In  its  most  complex  form,  the  eight 
basic  neurons  are  combined  into  a simultaneous,  interactive 
lateral  inhibition  network,  to  be  described  later. 

C.  THE  TARGET  NEURON 


NEUROE  is  the  computer  program  which  models  the  target 
neuron.  Target  neurons  in  visual  processing  are  probably  in 
the  retina,  lateral  geniculate  nucleus,  pons,  or  the  visual 
cortex,  and  perform  specific  tasks,  such  as  firing  for  one 
very  specific  visual  stimulus.  Neuron  B (Pig  4b)  represents 
a target  neuron  which  fires  when  its  E-inputs  coincide  with 
peaks  in  a sinusoidally  oscillating  PSP.  Such  a neuron  must 
have  the  oscillating  PSP  created  in  some  way;  in  neuron  B, 
this  is  done  by  E and  I-inputs  provided  by  outputs  from  an 
oscillator,  neuron  A.  There  are  reasons  for  postulating  the 
existence  of  an  element  such  as  neuron  B.  Pirst,  the 
freguency  of  oscillation  of  the  PSP  could  be  related  in  some 
way  to  the  frequencies  preferentially  present  in  the 
electroencephalogram  during  some  task  [Ref  11].  Second, 
sinusoidally  varying  PSP's  have  been  observed  in  nature  [Ref 
2].  In  this  thesis,  the  additional  E-inputs  come  from  the 
summed  outputs  of  the  eight  channels  of  the  lateral 
inhibition  program;  neuron  B has  copious  outputs  when  these 
additional  E-inputs  occur  at  the  right  frequency.  This 
condition  will  correspond  to  a small  target  crossing  the 
visual  field  at  the  preferred  angular  velocity. 
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FIGURE  5 - FLOW  CHART  OF  EIPSP 


0.  THE  NEURAL  OSCILLATOR 


NEUOSC  is  the  computer  program  which  models  a neuron 
(neuron  A,  Fig  4a)  which  generates  regularly  spaced  groups 
of  outputs.  These  outputs  are  then  used  to  drive  the  ESP  of 
target  neuron  B.  NEUOSC  is  versatile  in  that  it  allows  a 
choice  of  several  ways  of  producing  these  regularly  spaced 
output  groupings.  All  methods  are,  of  course,  compatible 
with  how  neurons  have  been  observed  to  function.  Neuron  A 
works  by  first  being  excited  to  threshold  and  producing 
output.  This  is  done  by  E-inputs  at  the  dendrites,  by  self 
excitability,  or  by  a combination  of  the  two.  The  outputs 
are  then  fed  back  as  inhibitory  inputs  after  a specifyable 
time  delay.  By  choosing  the  right  excitation  level, 
inhibitory  feedback  delay,  and  inhibitory  feedback  strength, 
the  neuron  can  be  made  to  provide  bunched  output  pulse 
trains  where  the  bunches  occur  at  any  desired  frequency. 

Self  excitability  has  been  observed  in  microelectrode 
studies  to  be  of  either  sawtooth  or  sinusoidal  form  [Ref  2]. 
All  programs  in  this  thesis  utilize  the  sawtooth  fora,  which 
is  generated  in  the  program  by  forming  a ramp  in  the  PSP 
block.  T%e  ramp  begins  following  a refractory  period  and 
ends  when  an  output  occurs.  The  slope  of  the  ramp  is 
callable  as  a parameter.  The  outputs  from  neuron  A are  used 
as  E-inputs  to  drive  neuron  B's  PSP,  while  the  inhibitory 
feedback  pulses  of  neuron  A are  used  as  the  I-inputs  to 
drive  neuron  B's  PSP. 

The  use  of  excitatory  as  well  as  inhibitory  driving 
allows  a much  higher  frequency  in  the  PSP  of  neuron  E.  By 
using  excitatory  inputs  alone  to  drive  the  PSP,  the 
frequency  at  which  a significant  amplitude  is  attainable  is 


liaited  by  the  exponential  fading  of  the  postsynaptic 
response.  By  alternately  driving  with  I-inputs,  the  sharp 
rise  of  the  inhibitory  postsynaptic  response  curve  is 
eaployed  to  guickly  drive  the  PSP  below  rest  potential,  thus 
speeding  up  the  oscillation. 

In  addition  to  the  noraal  modes  of  operation,  neuron  A 
is  capable  of  accepting  external  I-inputs,  and  also  of 
excitatory  feedback,  which  night  be  of  future  use  in 
aodeling  excessive  neural  discharge,  such  as  might  occur 
during  epilepsy. 


E.  ADAPTIVE  NEURON 

PSPFAT  is  a computer  program  which  models  the  behavior 
of  a neuron  whose  parameters  change  with  time,  or  adapts  to 
its  inputs. 

The  underlying  purpose  for  modeling  an  adaptive  neuron 
was  to  have  a neural  circuit  element  which  would  fire  for 
only  a brief  burst  following  a step  input  of  excitation. 
This  would  be  roughly  equivalent  to  a neural  differentiator, 
although  the  behavior  when  the  step  input  is  terminated  is 
inconsistent  with  this  analogy  (the  neuron  does  not  go  to  a 
negative  firing  rate  due  to  a negative  rate  of  change  of 
inputs).  The  adaptive  neuron  would  serve  as  a means  to 
separate  change  from  steady  state,  and  would  have  strong 
output  only  immediately  following  change.  Such  a neuron 
might  be  useful  in  a motion  detecting  network  where  a 
bright,  moving  spot  of  light  were  involved. 

Neurons  have  been  observed  which  have  a certain  low 
firing  rate  due  to  self-excitability  under  quiescent  (no 
input)  conditions  [Ref  2].  When  provided  with  an  excitatory 
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stiaulus,  the  firing  rate  is  at  first  rapid,  but  then  dies 
off  to  a level  only  slightly  above  the  quiescent  rate  (Pig 
6a)  . When  the  excitatory  stimulus  is  terminated,  firing 
returns  to  the  guiescent  rate.  When  an  inhibitory  stimulus 
is  turned  on,  the  firing  rate  goes  to  zero  initially,  but 
then  returns  to  a rate  slightly  below  the  guiescent  rate. 
Termination  of  the  inhibitory  stimulus  causes  the  firing 
rate  to  return  to  quiescent. 

Before  proceeding  with  PSPFAT,  a good  deal  of  thought 
was  put  into  exactly  what  physical  process  is  responsible 
for  the  phenomenon  being  modeled.  When  a neuron  is  referred 
to  as  less  excitable,  this  normally  means  the  membrane  is 
hyperpolarized;  being  more  excitable  normally  refers  tc  the 
membrane  being  depolarized.  These  conditions  are  fully 
modeled  by  the  basic  EIPSP  neuron  as  PSP  is  continuously 
computed,  and  do  not  seem  to  be  pertinent  to  modeling 
adaptation,  fatigue,  or  facilitation.  Adaptation  is  a rather 
nonspecific  term  meaning  that  the  neuron's  characteristics 
adapt  to  changing  conditions.  A change  in  spike  threshold 
should  not  be  entirely  ruled  out  as  a mechanism.  A change  in 
threshold  would  represent  a change  in  membrane 
characteristics,  and  the  real  answer  to  how  the  observed 
phenomena  are  caused  probably  does  involve  such  a change. 
But  it  is  more  likely  that  spike  production,  and  not  input 
history,  would  cause  membrane  changes  at  the  axon  hillock. 
There  is  no  simple  way  to  envision  the  spike 
product ion/thresho Id  change  scheme  resulting  in  the  behavior 
of  Pig  6a. 

Considerable  work  was  done  with  a neuron  model  which 
changed  its  threshold  in  response  to  inputs.  A history  of 
E-inputs  would  raise  the  threshold,  thus  reducing  the  firing 
rate.  Similarly,  a history  of  I-inputs  would  lower  the 
threshold.  The  object  here  was  to  make  the  inputs  less 
potent  as  they  became  more  numerous,  a law  of  diminishing 
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returns  cf  sorts.  That  object  was  not  really  met  by  the 
method  of  changing  the  threshold.  The  model  produced 
plausible  results  for  E-inputs,  and  less  believable  results 
for  I-inputs.  Because  only  a single  parameter  (spike 
threshold)  changed  in  order  to  give  the  desired  effect  for 
both  E and  I-inputs  histories,  the  algebraic  difference  (E 
minus  I)  was  decided  upon  to  apply  to  threshold  change. 
This  is  an  unrealistic  scheme,  since  no  such  E minus  I input 
dependence  of  threshold  is  present  in  nature.  The  results 
of  this  model  are  not  included  in  this  thesis. 

A more  likely  scheme  is  that  as  inputs  arrive  at  the 
presynaptic  site  of  neurotransmitter  release  in  great 
numbers,  the  neurotransmitter  is  at  first  plentiful,  but 
then  is  depleted,  accounting  for  the  loss  of  potency  in  the 
postsynaptic  response.  Alternately,  depletion  of  the  ATP 
supplies  used  by  the  synapse  as  energy  for  vessicle  release 
would  have  the  same  effect.  This  would  account  for  rhe 
behavior  of  Pig  6a,  but  would  not  explain  facilitation,  in 
which  a synapse  is  "primed"  by  the  arrival  of  a few  inputs 
such  that  the  postsynaptic  response  grows.  It  is  probable 
that  this  behavior  results  from  subsynaptic  membrane  changes 
rather  than  more  neurotransmitter  made  available 
presynaptically,  although  the  latter  has  not  been  ruled  out. 
Fatigue  could  also  result  from  presynaptic  inhibition, 
wherein  outputs  are  feb  back  to  presynaptic  terminals  as 
inhibitory  pulses,  thus  modulating  the  release  of 
neurotransmitter.  Evidence  also  exists  that  perhaps  repeated 
outputs  cause  buildup  in  the  extra-cellular  potassium  ion 
concentration,  and  that  this  buildup  modulates  the  release 
of  neurotransmitter  [Bef  16]. 

Fortunately,  it  is  unnecessary  to  chose  one  or  the  other 
when  modeling,  since  only  the  postsynaptic  response  is  of 
concern.  PSPFAT,  then,  varies  the  potency  of  E and  I-inputs 
by  varying  the  size  of  the  postsynaptic  response  according 
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to  the  running  average  of  inputs.  Thus  the  excitatory  and 
inhibitory  effects  can  be  handled  separately.  As  a mass  of  E 
or  I-inputs  occurs,  the  size  of  the  postsynaptic  response 
decreases,  thus  modeling  fatigue. 

In  the  FSPFAT  program,  this  task  is  handled  by  counting 
inputs  over  the  specified  time  interval  preceeding  each  word 
of  the  IX  block. 

Since  only  integer  arithmetic  is  used  in  neural  modeling 
(in  order  to  maximize  core  usage) , a small  reduction  in  the 
postsynaptic  response  strength  of  say  one-tenth  had  to  be 
handled  indirectly.  This  was  done  by  first  dividing  both  the 
E and  I postsynaptic  response  blocks  by  128,  and  then 
computing  (on  each  pass)  an  integer  for  building  them  back 
up  for  ESP  formation.  These  integer  multipliers  take  into 
account  the  running  averages  and  the  factor  by  which  the 
running  averages  affect  the  postsynaptic  responses. 
Specifically,  the  integer  for  building  the  PSP  blocks  back 
up  is  [128- (factor  X running  average)  ].  If  the  factor  is 
positive,  the  integer  will  be  smaller  than  128,  and  fatigue 
results.  By  selecting  a negative  factor,  synaptic 
facilitation  can  be  modeled. 


F.  LATEBAL  INHIBITION 


The  original  lateral  inhibition  program  is  a highly 
complicated,  intricate,  and  interactive  composition,  and  is 
a great  credit  to  LT.  Dennis  Marvel,  who  wrote  it  with 
suggestions  from  Dr.  Marmont.  When  the  author  inherited  it, 
however,  it  had  several  shortcomings,  which  will  be 
enumerated  as  they  occur. 

LINHIB  consists  of  eight  IK  data  blocks,  each 
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representing  a channel,  or  neuron,  such  as  a retinal 
ganglion  cell  (Fig  7) . LINHIB  models  any  lateral  inhibition 
network,  however,  and  is  not  necessarily  retinal.  Inputs  to 
the  network  are  from  upstream  neurons  (perhaps  bipolar 
cells)  which  synapse  on  each  of  the  eight  ganglion  cells. 
Outputs  are  sent  via  the  optic  nerve  to  the  brain  for 
further  processing.  Inhibition  results  whenever  an  output 
occurs.  An  output  from  a neuron  inhibits  its  neighbors,  but 
never  itself,  and  is  mediated  by  amacrine  and  horizontal 
cells  in  the  retina. 

The  original  version  of  LINHIB  produced  equal  inhibition 
in  all  seven  neurons  each  time  an  output  occurred.  This 
restraint  was  removed  by  the  addition  of  program  segments 
which  compute  the  separation  between  the  neuron  producing 
the  output  and  the  neuron  being  inhibited,  and  then  perform 
a table  lock-up  to  determine  the  appropriate  inhibitory 
strength.  Termed  the  lateral  inhibition  decay  (LID)  table,  a 
typical  entry  would  be  5432100.  The  first  number  describes 
the  strength  of  inhibition  of  the  neuron  adjacent  to  where 
the  output  occurred,  while  the  last  number  describes  the 
strength  cf  inhibition  of  the  neuron  most  remote  from  where 
the  output  occurred.  Therefore,  this  LID  describes  decay  in 
relative  inhibitory  strength  as  distance  from  the  cutput 
neuron  increases.  Later,  as  more  flexibility  was  desired,  a 
two  sided  LID  table  was  devised,  allowing  inhibitory 
influence  to  decay  differently  in  different  directions.  A 
typical  two-sided  (but  symmetrical)  LID  table  might  be 
001234505432100,  with  the  central  zero  always  present  to 
indicate  no  self-inhibition.  Thus  LID=000000005432 100  would 
produce  a network  in  which  inhibition  occurred  to  one  side 
only. 

Originally,  all  inhibition  occurred  after  a fixed  delay 
in  time  (or  really,  words  of  the  block).  In  order  to  add 
more  flexibility,  this  was  changed  to  allow  a linear  delay 
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in  the  spread  of  inhibition.  That  is,  if  the  neuron  adjacent 
to  the  output  is  inhibited  after  0.001  seconds,  then  the 
next  would  be  inhibited  after  0.002  seconds,  the  next  after 
0.003  seconds,  and  so  on  (Fig  8) . This  linear  delay  could  be 
accoaplisbed  through  synaptic  delays,  as  shewn. 

The  LINHIB  program  works  in  ouch  the  same  way  as  the  PSP 
production  in  the  basic  E1PSP  model.  That  is,  the  EPSR  and 
the  IPSR  are  generated  in  a pair  of  IK  blocks  which  are  aged 
and  sunned  the  proper  number  of  times  to  form  each  PSP  word. 
The  biggest  difference  is  in  the  way  this  is  done.  Because 
LINHIB  models  eight  neurons  vice  a single  neuron,  much  more 
memory  is  reguired.  It  would  have  been  impossible  to  fit 
eight  E-input  blocks,  eight  I-input  blocks,  eight  output 
blocks,  eight  PSP  blocks,  and  two  PSR  blocks  all  into  the 
core  available.  Therefore,  the  E-inputs,  outputs,  and 
I-inputs  are  all  stored  in  compressed  form.  The  E-input  and 
output  blocks  are  each  256  word  blocks  containing  the 
addresses  of  the  markers.  The  I-inputs  are  stored  in  a 2K 
block  where  a given  inhibitory  unit  consists  of  one  word  of 
address  and  cne  word  of  relative  strength. 

Processing  begins  by  stepping  through  word  number  one  of 
each  of  the  eight  data  blocks,  and  polling  the  compressed 
E-input  block  to  find  an  address  match,  which,  if  found, 
signals  an  E-input  at  that  location  (Fig  9) . Each  time  an 
E-input  is  found,  a zero  is  placed  on  the  E-age  stack.  The 
zero  indicates  that  the  PSR  has  not  aged  at  all,  since  the 
input  has  just  occurred.  Also,  an  E-input  counter  is 
incremented.  Next,  the  PSP  word  is  formed  by  adding  the 
proper  number  of  properly  aged  PSR's  to  the  PSP  block,  When 
word  number  one  of  all  eight  data  blocks  is  processed,  a 
similar  procedure  is  followed  for  inhibitory  inputs.  The 
difference  here  is  that  the  compressed  I block  is  different 
in  format;  the  I address  is  compared  with  the  current 
processing  address  to  detect  a match.  Then,  the  inhibitory 
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strength  is  used  to  increment  the  I address  for  more 
■atching.  Thus  if  the  relative  inhibitory  strength  is  five, 
this  would  have  the  effect  of  producing  five  inhibitory 
pulses. 

Following  the  subtraction  of  the  proper  number  of 
properly  aged  IPSH's  from  the  PSP  word  in  each  of  the  eight 
blocks,  a comparison  of  PSP  with  spike  threshold  is  made.  If 
PSP  eguals  or  exceeds  spike  threshold,  the  proper  address  is 
entered  in  the  compressed  output  block,  and  the  task  of 
placing  inhibition  is  begun  (this  has  already  been 
described).  Finally,  processing  steps  to  the  next  word  and 
returns  to  the  top  of  the  flow  graph. 

After  much  work  had  been  done  with  the  LINHIB  program, 
the  unsettling  discovery  was  made  that  many  of  the  results 
were  probably  invalid  because  of  overflowing  the  capacity  of 
the  stack  which  is  used  to  age  the  inhibitory  inputs.  As 
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mentioned  earlier,  when  graded  inhibition  was  added  to  the 
program,  this  was  done  by  providing  more  markers  for 
stronger  inhibition,  and  fewer  markers  for  weaker 
inhibition.  The  actual  inhibitory  effect  was  then  set  by 
adjusting  the  value  of  the  IPSR.  The  end  result  of  multiple 
markers  was  to  fill  and  in  some  cases  overflow  the  stack, 
thus  invalidating  many  earlier  runs.  This  problem  was  solved 
by  enlarging  the  E and  I-age  factor  stacks  from  IK  to  6K  in 
size.  The  extra  core  memory  was  gained  by  rewriting  the 
entire  program  in  the  APTEC  "language."  Additionally, 
program  steps  were  added  which  check  the  compressed  E-input, 
I-input,  and  output  blocks,  and  E/l-age  stacks  for  overflow, 
and  warn  the  operator  of  these  undesirable  conditions. 

Another  pcint  of  concern  which  turned  out  to  be  entirely 
cosmetic  involved  the  placement  of  inhibition  after  the  ramp 
delay  was  added.  Because  the  eight  processing  blocks  are 
continuous  in  core,  an  output  near  the  end  of  one  block  will 
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produce  inhibition  at  addresses  which  are  beyond  the  end  of 
the  block  intended,  but  which  fall  near  the  beginning  of  the 
following  block.  It  turned  out  that  these  bogus  1-pulses  do 
not  get  taken  as  inputs  for  PSP  computation,  because  by  the 
time  they  get  placed  in  the  compressed  I-block,  processing 
is  well  beyond  their  location.  They  do  get  displayed, 
however,  but  this  problem  is  easily  solved  by  graphical 
methods. 


G.  INLIE  ANE  OOTLIB 


The  various  IK  blocks  of  input  pulse  trains  are  built  by 
a program  called  INLIB.  The  operator  selects  one  of  two 
versions  of  random  generation,  one  of  two  versions  of 
constant  freguency,  or  frequency  modulation.  These  are 
stored  on  disk  uncompressed  for  use  with  any  program  except 
LINHIB,  which  reguires  a 256  word  compressed  E-input  block. 
The  formation  of  this  compressed  block  for  LINHIB  is  by  a 
program  called  OUTLIB,  which  reads  any  eight  IK  blocks  from 
disk  and  compresses  them  by  storing  marker  addresses  in  the 
256  word  block  which  is  then  stored  on  disk.  The  author 
rewrote  these  programs  in  APTEC,  improved  the  freguency 
modulation  program's  flexibility,  wrote  one  new  program  for 
generating  constant  freguency  blocks,  made  program 
modifications  which  increased  the  total  number  of  inputs 
which  could  be  placed  in  the  256  word  block,  and  generated 
numerous  additions  to  the  existing  library  of  inputs. 
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IV.  RESULTS 


1.  CONVENTIONS  FOB  PLOTTING  OF  DATA 

There  are  several  clarifying  points  which  will  aid  the 
reader  in  understanding  the  modeling  plots.  1)  should 
excitatory,  inhibitory,  or  output  spike  Barkers  appear  which 
are  of  differing  heights,  this  is  of  no  account.  The 

prograas  siaply  test  for  the  presence  of  a non-zero  Barker, 
not  caring  about  its  aagnitude.  2)  Not  all  plots  were  aade 
for  every  run  for  LINHIB,  as  this  would  reguire  two  or  three 
full  pages.  Rather,  only  those  plots  needed  to  illustrate 
the  discussion  are  included.  In  general,  the  PSB's,  the 
PSP's,  the  inputs,  and  the  outputs  are  included.  3)  The 
sizes  of  the  IPSR  and  the  EPSR  relative  to  each  other  are 
accurate,  but  due  to  the  consideration  of  Baking  as  auch 
detail  as  possible  in  the  space  available,  they  will  not 
always  be  scaled  the  same  as  the  PSP  plot.  Scaling  is, 

however,  dene  after  all  conputations  are  conplete,  and  in  no 
way  affects  the  accuracy  of  the  results.  4)  The  plots  each 
cover  a tine  course  of  100  Billiseconds.  5)  The  difference 
between  threshold  and  resting  neabrane  potential  is  taken  to 
be  20  Billivclts.  This  can  be  gauged  by  noting  that  upon 
occurrence  of  an  output,  the  PSP  trace  drops  vertically  froa 
threshold  to  rest  potential  and  reaains  there  for  at  least 
the  duration  of  the  refractory  period.  All  plots  use  20 

aillivolts  as  spike  threshold.  6)  Multiple  Barkers  in 
inhibitory  plots  are  understood  to  be  an  indication  of 

strength  of  inhibition,  and  not  actually  aultiple  inhibitory 
action  potentials.  (7)  All  plots  are  the  results  of 
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computer  calculations  Bade  with  the  nodeling  prcgraas 
descrioed.  The  plots  are  therefore  not  terely  how  the  author 
thinks  things  should  happen,  but  how  coaputer  calculations 
indicate  they  should  happen. 


B.  TEMPORAL  PHENOMENA 

It  is  iapossible  to  really  separate  "spatial"  froa 
"teaporal"  when  dealing  with  a spatial  neural  array 
processing  a tiae  seguence  of  events,  but  the  distinction 
drawn  here  is  just  as  the  reader  aight  expect:  those 

situations  in  which  a change  in  tiae  are  predoainantly 
iaportant  are  covered  under  teaporal,  whereas  if  the  change 
froa  neuron  to  neuron  is  aost  iaportant,  it  is  terned 
spatial. 

1-  fatigue  and  Facilitation 


The  first  teaporal  phenoaena  to  be  aodeled  are 
fatigue  and  facilitation. 

Figure  10  shows  a neuron,  initially  quiescent, 
receive  a train  of  E-inputs.  As  the  E-input  history 
increases,  the  PSfi  due  to  each  E-input  becoaes  saaller.  This 
can  be  seen  by  observing  the  PSP  trace  carefully.  At  first 
each  E-input  produces  a sizeable  response.  But  as  E-inputs 
accuaulate,  the  PSR's  becoae  so  snail  that  they  are  barely 
perceptable  atop  the  guiescent  ranp.  As  discussed  earlier, 
this  behavior  probably  results  either  froa  neurotransaitter 
depletion  or  subsynaptic  aeabrane  changes.  Figure  11  shows 
fatigue  in  the  case  of  Z-inputs.  The  output  trace  clearly 
shews  an  initial  extinction  and  a gradual  return  to  firing 
at  just  below  the  guiescent  rate. 
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Ey  selecting  a negative  number  for  the  fatigue 
factor  in  PSPFAT,  facilitation  can  be  shown  (Fig  12) . 
Initially  guiescent,  the  neuron  is  excited  by  the  now 
familiar  E-input  train.  As  E-inputs  accumulate,  some 
submicroscopic  effect  causes  the  PSB  for  each  to  wax  in 
strength,  resulting  in  an  accelerating  output.  The 
facilitation  modeled  here  is  of  very  short  duration,  and 
should  not  be  confused  with  any  long  term  changes  which 
result  from  the  plastic  nature  of  synapses  [Ref  7].  Such 
long  term  synaptic  facilitation  is  a likely  mechanism  for 
long  term  memory.  The  very  short  duration  facilitation  of 
Fig  12  could  conceivably  be  responsible  for  a person  being 
able  to  glance  at  a table  full  of  objects,  look  away,  and 
picture  the  scene,  enabling  him  to  name  the  objects,  whereas 
after  several  seconds,  this  is  no  longer  possible. 

One  uncertainty  in  the  fatigue  models  is  that  the 
real-life  phenomenon  might  have  resulted  from  receptor 
adaptation  rather  than  from  synaptic  changes  or  subsynaptic 
membrane  changes.  The  E or  I-input  train  used  in  the  models 
was,  in  most  experimental  measurements,  actually  a light  dot 
or  surround  stimulus  rather  than  a known  pulse  train  of 
constant  frequency.  Therefore,  if  a constant  intensity  light 
stimulus  had  resulted  in  receptor  adaptation,  the  proper 
representation  of  inputs  to  the  neuron  would  be  a train  of 
decreasing  frequency.  This  alone  could  account  for  the 
decreasing  output  rate  after  the  stimulus  had  been  on  for  a 
short  time.  The  light  and  dark  adaptation  of  receptor  cells 
is  a well  known  phenomenon,  but  this  occurs  with  a time 
constant  on  the  order  of  minutes,  and  could  not  result  in 
changes  on  the  order  of  one  second  or  less.  A review  of 
individual  receptor  responses  in  the  frog  retina  to  light 
stimuli  [Bef  15]  indicates  no  discernable  adaptation  over  a 
period  cf  one  second.  This  may  not  be  true  of  all  retinal 
receptors,  however. 
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FIGURE  10  - FATIGUE  OF  E-INPUTS 


FIGURE  12  - FACILITATION 


2.  She  Neural  Oscillator 


The  neural  oscillator  is  used  to  generate  bunched  E 
and  I-pulse  trains  for  use  with  the  target  neuron.  The 
character  of  these  bunched  pulse  trains  can  be  observed  in 
the  E and  I-input  plcts  of  Pig  13b. 

One  ncde  of  generating  the  required  outputs  consists 
of  allowing  the  oscillator  neuron  to  have  a guiescent  firing 
rate,  and  then  feeding  back  the  outputs  as  inhibitory  inputs 
(Pig  13a)  . By  then  using  the  output  and  feedback  pulse 
trains  as  driving  inputs  to  neuron  B,  the  oscillatory  PSP  is 
created  (Fig  13b) . Alternately,  the  oscillator  neuron  can  be 
provided  with  random  E- inputs  (Pig  14a) . When  outputs  occur 
they  are  fed  tack  as  I-inputs,  as  before. 

Although  instability  in  the  oscillator  neuron  is  not 
of  importance  to  other  work  in  this  thesis,  a demonstration 
is  included  to  show  the  versatility  of  the  NEUOSC  program 
(Pig  15)  . Note  that  originally,  outputs  result  only  frcm  the 
quiescent  firing  rate.  Since  the  outputs  are  fed  back  as  E 
vice  I-inputs,  the  outputs  wax  in  intensity  at  an  ever 
increasing  rate  until  the  neuron  fires  at  near  the  maximum 
allowed  by  the  refractory  period. 


3*  The  Target  Neuron 


The  cutputs  of  the  oscillator  neuron  are  used  to 
create  a sinusoidally  varying  PSP  in  the  target  neuron.  The 
target  neurcn  would  then  fire  most  strongly  in  response  to 
E-inputs  at  the  proper  phase  and  frequency.  Conversely,  if 
the  E-inputs  were  of  random  character,  the  target  neuron 
could  produce  a very  regular  output  of  known  frequency  and 
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phase 


A good  example  of  a higher  frequency  sinusoidal  PSP 
is  shown  in  Fig  16.  Note  the  role  of  alternating  E and 
I-inputs  in  driving  the  PSP  in  opposite  directions.  If  an 
oscillating  ESP  were  created  using  E-inputs  alone,  the  slow 
decrease  of  the  PSR  would  cause  a gradual  depolarization 
toward  spike  threshold,  placing  a severe  upper  limit  on  the 
frequencies  and  amplitudes  attainable.  The  idea  that  higher 
frequencies  may  be  used  in  CNS  processing  is  being  pursued 
in  electroencephalogram  work  by  Dr.  Marmont  and  his  students 
in  the  70-95  hertz  range.  It  is  probably  an  insult  tc  the 
CNS  to  believe  it  incapable  of  useful  activity  at 
freguencies  above  ten  hertz  [Ref  11]. 

Figure  17  is  an  example  of  regularization  of  random 
inputs.  A basic  constraint  is  that  the  PSP  driving  inputs 
should  be  much  more  numerous  than  the  random  inputs,  cr  the 
latter  will  tend  to  dominate  the  PSP  characteristics. 
Alternately,  providing  a weaker  PSR  for  the  random  inputs 
would  allcw  the  driving  inputs  to  dominate  the  PSP,  even 
though  the  random  inputs  might  be  very  numerous. 


4 . 3211211  Detection 

Notion  detection  systems  have  been  demonstrated  by 
microelectrode  studies  in  many  species.  These  studies  tend 
to  indicate  that  motion  detection  is  performed  by  neurons 
having  very  specific  characteristics  as  to  velocity, 
direction,  and  fora  for  optimum  detection.  The  adaptive 
advantage  of  an  efficient  system  for  motion  detecticn  is 
obvious:  the  objects  in  the  visual  field  which  are  of 
greatest  interest  to  an  animal  in  the  wild  are  those  which 
are  in  acticn. 


PSP 


OUTPUTS 


A.  OSCILLATOR  NEURON 


EPSR 


ADDITIONAL  INPUTS 


B.  TARGET  NEURON 


FIGURE  13  - SINUSOIDAL  PSP,  SPONTANEOUS 
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FIGURE  15  - INSTABILITY 
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Eand  Pass 


The  tacget  neuron  with  oscillatory  PSP  can  be 
put  to  practical  use  as  a target  velocity  band  pass  gate.  A 
tiny  target  crossing  the  LINH1B  eight  neuron  array  is 
represented  ty  successive  inputs  at  each  neuron  (Figs  18  and 
19) . The  outputs  are  applied  to  the  target  neuron,  where 
the  PSP  is  oscillating  (Fig  20b) . Here,  only  E-inputs  froa 
the  neural  cscillator  (Fig  20a)  have  been  used  to  create  the 
pxoper  freguency.  Because  the  inputs  to  neuron  B free  the 
LINHIB  network  are  at  the  proper  freguency  and  phase,  strong 
outputs  are  produced  froa  neuron  B.  This  signal  sight  say  to 
the  cortical  centers  for  visual  association  that  there  is  a 
target  in  the  receptive  field  of  neuron  B which  has  X 
radians  per  second  of  angular  velocity.  The  organisa  could 
then  take  action  appropriate  to  the  stimulus. 

Neuron  B is  a suaaing  neuron,  since  all  eight 
channels  of  11HH1B  converge  to  excite  neuron  B.  The  sussing 
is  perforaed  only  if  the  inputs  occur  at  the  proper  points 
on  the  PSP  oscillation. 

The  fact  that  phase  as  well  as  freguency  oust  be 
correct  in  cxder  for  the  speed  band  pass  gate  neuron  to  pass 
the  signal  is  a sticky  point  which  has  not  been  fully 
solved.  Perhaps  the  appearance  of  a target  causes  a strobing 
effect  in  the  oscillating  neuron  which  autoaatically  adjusts 
the  PSP  oscillations  to  optiaua  phase.  Alternately,  perhaps 
a given  freguency  has  four  or  sore  target  neurons  such  that 
any  phase  of  target  would  cause  outputs  fros  one  or  sore 
target  neurens.  Creating  sultiple  target  neurons  with 
oscillatory  PSP's  of  different  phase  is  a simple  matter  of 
delaying  the  driving  E and  I-input  pulse  trains  by  a 
fraction  of  the  oscillatory  period,  as  shown  in  Fig  2b. 


Ihere  is  evidence  that  neurons  do  exist  which 
are  specific  for  speed  of  target  notion  [Refs  1 and  8], as 
well  as  for  direction.  Pontine  cells  respond  to  a band  of 
speeds,  whereas  retinal  speed  specificity  tends  to  be  less 
discrete  than  a narrow  band  pass  gate  night  provide.  That 
is,  in  the  retinal  ganglion  cells  of  the  rabbit,  response 
tended  to  be  broken  down  into  two  crude  groups:  high  speed 
detectors  and  low  speed  detectors. 

b.  Hull  versus  Preferred  Direction 


Direction  specificity  is  now  nodeled  using  the 
lateral  inhibition  network.  In  Figs  18  and  19  the  lateral 
inhibition  cetwork  was  used,  but  the  paraneters  were 
adjusted  so  that  the  lateral  nature  of  the  network  was 
suppressed.  In  other  words,  neuron  S has  an  early  output, 
and  produces  inhibition  in  its  neighbors.  But  this 
inhibition  is  no  longer  present  (PSP  has  returned  tc  the 
resting  value)  in  neuron  T when  E-inputs  representing  the 
target  reach  it.  Obviously,  if  the  duration  of  the  PS  6 for 
I-inputs  were  longer,  a aoving  target  could  be  blanked  out. 
Furtheraore,  if  inhibition  in  one  direction  were  ouch 
stronger  than  inhibition  in  another  direction,  target  notion 
in  one  direction  could  be  detected  while  target  motion  in 
the  other  direction  would  be  blanked  out.  Figures  21,  22, 
and  23  show  the  results  of  this  modeling.  Since  the  LID 
table  is  000000005432100,  the  upward  target  is  not  inhibited 
at  all,  and  each  neuron  produces  an  output.  The  downward 
target,  however,  produces  only  two  outputs,  in  neurons  S and 
X.  When  neurcn  S fires,  inhibition  spreads  downward  and 
hyperpolaxizes  the  aembranes  of  the  neurons  below  it. 
Because  a long  duration  IPSR  was  used,  outputs  are  prevented 
in  the  four  neurons  below  neuron  S.  Neuron  X is  both  later 
in  tiae,  and  had  weaker  inhibition  to  begin  with,  so  it 
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produces  an  output  when  excited.  As  with  neuron  S,  the 
output  from  neuron  X is  effective  in  preventing  outputs  from 
neurons  Y and  Z.  Thus  upward  is  the  so-called  preferred 
direction,  while  downward  is  the  null  direction. 

The  target  neuron  is  once  again  a summer.  For 
all  circuits  except  band  pass,  it  is  a simple  summer,  where 
one  input  is  generally  sufficient  to  produce  an  output.  For 
band  pass,  frequency  and  phase  of  inputs  must  match 
frequency  and  phase  of  the  oscillatory  PSP  in  order  for 
potent  summing  to  occur. 

There  are  important  differences  between  this 
model  for  null  versus  preferred  directions  and  that  of  Kef 
12  (Fig  24) . The  density  of  receptors  in  the  retina  evolved 
to  a very  high  value  for  very  good  reasons,  namely 
sensitivity  and  acuity.  More  receptors  means  that  many 
receptors  can  (via  bipolar  cells)  feed  a single  ganglion 
cell,  resulting  in  very  good  sensitivity.  Bore  receptors 
also  means  that  visual  acuity  improves,  sensitivity  being 
constant.  The  neural  circuit  of  Fig  24,  ignoring  the  dashed 
connections,  has  the  disadvantage  of  using  precious 
receptors  for  a dedicated  purpose.  This  would  degrade 
sensitivity  and  acuity,  receptor  density  held  constant.  The 
model  used  in  this  thesis,  on  the  other  hand,  has  all 
receptors  synapsing  in  the  "normal"  manner,  and  uses  lateral 
inhibition  among  the  ganglion  cells  to  derive  the  .null  and 
preferred  directions*  Furthermore,  the  lateral  inhibition 
network  could  be  located  extraretinally , such  as  in  the 
lateral  geniculate  body  or  the  visual  cortex,  although  this 
would  not  account  for  the  observed  directional  specificity 
of  retinal  ganglion  cells. 

A more  careful  consideration  of  Fig  24  suggests 
that  each  group  cf  four  receptors  might  provide  outputs  to 
more  than  a single  cell.  This  would  make  more  economical  use 
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of  the  receptors,  and  could  result  in  a scheme  in  which 
acuity  were  not  degraded.  In  fact,  if  a second  set  of 
horizontal,  bipolar,  and  ganglion  cells  were  concurrently 
receiving  receptor  outputs,  but  with  "reverse"  organization, 
the  opposite  preferred  direction  could  be  derived.  The 
reverse  organization  is  also  shown  in  Fig  24,  but  smaller 
and  connected  by  dashed  lines.  This  scheme  represents  a 
considerable  increase  in  economy  over  the  previous  circuit. 

The  neural  circuit  of  Fig  24  does  have  the 
advantage  of  an  improved  nulling  in  the  null  direction.  The 
reason  for  this  is  that  the  target  hyperpolarizes  each 
bipolar  cell  before  exciting  it.  With  the  model  of  this 
thesis,  an  output  must  occur  before  inhibition  can  cccur. 
Thus  the  null  direction  can  never  have  a true  null.  This 
seems  less  than  tragic  when  a possible  use  of  such  direction 
specific  cells  is  considered.  Presumably  such  cells  ccme  in 
pairs.  That  is,  for  a cell  having  upward  null  and  downward 
preferre  1,  there  should  also  be  a cell  having  upward 
preferred  and  downward  null  directions  serving  the  same 
receptive  field.  Then,  in  addition  to  providing  signals  to 
the  visual  association  area  of  the  cerebral  cortex,  these 
neurons  might  be  responsible  for  proper  initiation  of  a 
reflex  such  as  eye  movement  for  target  tracking.  Thus,  so 
long  as  the  output  from  the  neuron  having  its  preferred 
direction  in  the  actual  direction  of  target  motion  -were 
substantially  stronger  than  the  neuron  which  should  be 
nulled,  the  proper  reflex  action  should  result. 
Additionally,  because  visual  processing  usually  occurs  in 
stages,  perhaps  the  nulling  could  be  improved  in  follow-on 
processing  stages. 

Ey  sending  the  output  of  the  null/preferred 
direction  network  to  a target  neuron  with  an  oscillatory 
PSP,  the  result  is  an  overall  circuit  capable  of  firing 
strongly  only  when  a target  in  a specific  direction  at  a 
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specific  speed  is  present,  as  might  be  found  in  the  pons. 
Such  fine  grained  speed  differentiation  has  not  been 
observed  retinally,  where  only  high  versus  low  speed 
detectors  have  been  proven  to  exist. 


c.  Fast  Pass 


The  next  model  designed  was  a network  of  the 
type  found  in  the  retina  of  the  rabbit,  which  would  detect 
only  fast  targets.  The  lateral  inhibition  network  of  Pig  6 
can  be  used  very  nicely  to  detect  only  fast  targets. 
Pigures  25  and  27a  rget  in  this  network.  The  heart  of  the 
high  speed  pass  behavior  of  this  network  is  the  fact  that 
the  inhibition  of  neighboring  neurons  is  delayed  in  ramp 
fashion  instead  of  the  usual  constant  delay.  This  can  be 
seen  by  observing  the  spread  of  inhibition  in  Figure  27a.  k 
target  which  is  fast  enough  is  able  to  excite  neuron  S to 
produce  an  output,  and  then  excite  neuron  T to  output  before 
neuron  T is  inhibited  by  neuron  S.  This  network  has  a very 
abrupt  velocity  dividing  line  between  detection  and  no 
detection. 

/ 

A slow  target  illuminating  the  high  pass  network 
is  depicted  in  Figs  26  and  27b.  Here,  the  inhibition 
resulting  from  the  output  of  neuron  S spreads  to  neurons  T, 
0,  V,  a,  and  X in  time  to  inhibit  them  before  the  arrival  of 
the  target  illumination.  Neuron  T is  beyond  the  influence  of 
inhibition  by  neuron  S,  and  therefore  has  an  output.  This 
network  could  be  made  more  effective,  then,  by  altering  the 
LID  table  such  that  inhibition  would  spread  farther. 

In  Fig  27,  note  that  the  ramp  delay  in  spread  of 
inhibition  consists  of  incrementel  delays  of  about  ten 
milliseconds.  This  could  be  accomplished  by  several  synaptic 
delays  [ Bef  6 ]. 
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C.  INHIBITION,  NULL 

FIGURE  23  - NULL  VERSUS  PREFERRED  DIRECTION,  PART  3 


d.  Slow  Pass 


IINHIB  can  also  be  used  to  aodel  a network  which 
detects  slow  targets  but  fails  to  detect  fast  targets.  This 
is  done  by  using  a constant  delay,  or  perhaps  a snail  rasp 
delay,  and  by  carefully  selecting  the  1PSB  duration.  The 
plots  for  such  a slow  pass  network  for  both  slow  and  fast 
targets  are  shown  in  Figs  28,  29,  and  30.  First,  for  the 
slow  target,  the  inhibitory  influence  on  neuron  T due  to  the 
outputs  froa  neuron  S are  largely  gone  by  the  tine  the 
target  illuainates  neuron  T,  and  an  output  results.  Only 
neuron  Z fails  to  fire,  but  it  is  probable  that  this  neuron 
would  fire  given  several  nore  Billiseconds,  judging  froa 
coaparisons  cf  neuron  Z's  PSP  with  that  of  neuron  7. 
Therefore,  let  us  say  that  all  eight  neurons  fired  or  would 
have  fired.  A high  speed  target  is  now  used  to  illuainate 
the  sane  network  (Fig  29) . Note  that  here,  only  five  of 
eight  neurons  fired  for  a speed  ratio  of  two  to  one.  This 
aay  seen  a rather  nodest  result,  but  consider  that  in  a 
single  pole  filter,  a frequency  ratio  of  two  to  one  would 
produce  an  attenuation  of  only  six  decibels,  which  is 
ccaparable  to  the  five  to  eight  ratio.  A subtle  additional 
fact  is  that  a double  speed  target  would  produce  rcughly 
half  the  nuaber  of  inputs  per  channel  to  the  network.  Iaking 
"credit"  for  this  result  of  high  speed,  the  characteristics 
of  the  network  iaprove  soaewhat  (Fig  30) . Here  only  three 
neurons  fire,  as  opposed  to  five  before. 

5.  Isaalatilfiiias  Randoa  Inputs 

The  noraal  approach  to  aodeling  is  to  deteraine  what 
hard  inforaation  exists  about  the  systea  to  be  aodeled, 
build  the  aodel,  and  then  use  it  to  deteraine  its  usefulness 
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in  duplicating  the  behavior  of  the  real  systea.  Once 
confidence  in  the  aodel  has  escalated  somewhat,  the  ncdeler 
can  begin  to  extend  application  to  areas  not  necessarily 
observed  in  real  life.  Then  if  an  interesting  output  results 
froa  the  aodel,  two  possibilities  exist.  First,  the  aodel 
aight  really  not  be  an  adequate  description  of  the  real 
systea,  and  the  ineresting  result  worthless.  Second,  the 
aodel  could  be  valid,  and  the  result  could  be  worth  looking 
for  in  the  real  systea. 

One  such  result  observed  while  using  the  lateral 
inhibition  network  with  randoa  inputs  is  that  an 
unrandomized  output  can  be  produced  (Figs  31  and  32)  . The 
suaned  output  plot,  while  not  completely  regular,  is 
nonetheless  much  more  regular  than  any  of  the  inputs.  Such 
orderliness  results  froa  the  very  character  of  the  lateral 
inhibition  network.  Namely,  all  channels  are  vying  for 
output  production.  When  an  output  occurs,  all  other  nearby 
channels  are  inhibited  for  the  duration  of  the  IPSR.  This 
aeans  that  during  this  interval,  nearby  outputs  are 
unlikely.  Since  this  interval  would  tend  to  be  the  same 
after  each  output,  the  result  is  a rather  uniform  time 
between  outputs.  Sore  complete  regularization  would  occur 
if  the  inhibitory  influence  were  to  spread  farther.  In 
support  of  this  notion,  note  that  the  two  closest  outputs 
(the  third  and  fourth)  in  the  summed  output  plot  originated 
from  spatially  distant  channels.  If  inhibitory  spread  had 
been  more  complete,  then  in  all  likelihood,  the  fourth 
output  would  not  have  occurred  so  soon. 

This  method  of  regularizing  random  inputs  is  to  be 
contrasted  with  that  of  Fig  17,  where  an  oscillatory  PSP  was 
needed. 


There  are  possible  uses  for  an  orderly,  regularly 
spaced  spike  train.  In  the  next  section  it  will  be  shown 


C.  OUTPUT 


FIGURE  28  - SLOW  PASS  DETECTOR,  SLOW  TARGET 
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B.  OUTPUT 

FIGURE  32  - REGULARIZATION  BY  LINHIB,  PART  2 
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that  a regularly  spaced  set  of  inputs  to  an  inhibitory 
network  has  extreme  phase  sensitivity,  so  long  as  the 
channels  have  roughly  egual  firing  rates.  That  is,  the 
channel  whose  input  phase  allows  it  to  produce  the  first 
output  will  gain  and  maintain  dominance  over  the  other 
channels.  This  could  be  considered  as  a possible  mechanism 
for  a person  being  able  to  direct  his  attention  to  only  one 
of  several  sensory  inputs  of  egual  level. 


C.  SPATIAL  PHENOMENA 


1.  Basic  Spatial  Behavior  of  LINHIB 


As  a basic  example  of  the  operation  of  LINHIB, 
consider  Figs  33  and  34.  The  inputs  are  random,  but  are  most 
dense  in  channel  V,  decreasing  in  either  direction.  It  is 
not  surprizing  that  more  outputs  occur  in  channel  V,  since 
more  inputs  were  provided  there.  But  more  importantly,  the 
ratio  of  outputs  to  inputs  is  highest  for  channel  V,  because 
of  lateral  inhibition.  That  is,  channel  V gains  dominance 
over  the  other  channels  by  having  outputs  which  inhibit 
them.  Thus  it  is  more  difficult  for  the  other  channels  to 
produce  outputs. 


2.  Inhibition  agd  Disinhibition 


The  two  most  basic  spatial  phenomena  which  can  be 
demonstrated  using  the  lateral  inhibition  network  are 
inhibition  and  disinhibition  (Figs  35  and  36).  Note  that 
only  three  of  the  eight  neurons  have  inputs,  and  that  at 
first*  only  two  have  inputs.  Neuron  X gains  dominance  over 
■seres  Z by  producing  an  output  first.  Because 
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LID=0Q0234505432 QO 0,  neuron  Z is  strongly  inhibited  and  does 
not  fire,  allowing  neuron  X to  maintain  dominance.  This 
describes  inhibition  of  neuron  Z,  and  would  be  analagous  to 
a microelectrode  measurement  neuron  Z while  providing  a 
bright  surround  stimulus. 

* 

When  neuron  0 is  illuminated  with  an  even  brighter 
stimulus  than  that  for  neuron  X,  neuron  0 gains  dominance 
and  inhibits  neuron  X.  This  results  in  a complete  lack  of 
inhibition  tc  neuron  Z,  which  now  fires.  (Inhibition  from 
neuron  0 spreads  only  as  far  downward  as  neuron  Y)  . 

Disinhibition  has  been  demonstrated  in  the  limulus 
eye  [Ref  6],  but  is  not  important  to  the  organism  in  its  own 
right.  It  is,  rather,  a result  of  the  lateral  inhibition 
network's  characteristics,  and  served  to  demonstrate  the 
spatial  extent  of  inhibitory  spread. 

3 . Bright  Line  Phenomena 


a.  Line  Sharpening  by  Inhibition 


One  function  which  a lateral  inhibition  network 
performs  in  vision  is  the  sharpening  of  lines  which  become 
blurred  due  to  scattering  in  the  eyeball's  structures.  The 
optics  of  the  eye  are  excellent,  but  are  not  perfect. 
Lateral  inhibition  is  a viable  method  for  improving  the 
visual  system  by  increasing  its  resolving  power  by 
performing  sharpening. 

A single,  narrow,  bright  line  against  a dark 
background  was  chosen.  A lateral  inhibition  network  will 
also  perfcrm  sharpening  on  a narrow  dark  line  on  a bright 
background,  probably  just  as  effectively  as  the  former  case. 
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But  since  the  eight  neuron  linear  array  is  tacitly  flanked 
by  many  other  neurons  in  total  darkness  (no  inhibition  is 
fed  into  the  network  from  above  neuron  S or  below  neuron  Z) , 
the  central  bright  line  was  the  most  reasonable  choice. 

An  important  point  is  that  line  sharpening 
occurs  whether  or  not  optical  blurring  occurs.  In  other 
words,  it  is  only  the  photon  distribution  on  the  retinal 
receptors  and  the  exact  nature  of  the  neural  connections 
which  determine  the  output  to  the  brain.  Neither  the  retina 
nor  the  brain  has  any  way  of  knowing  whether  a blurred  line 
distribution  on  the  retina  is  an  optically  blurred  image  of 
an  actual  sharp  line,  or  a perfect  image  of  an  actual 
blurred  line.  That  is,  a lateral  inhibitory  network  will 
sharpen  an j line,  whether  it  is  actually  sharp  or  not. 

line  sharpening  is  illustrated  in  Pig  37.  For 
this  run,  the  strength  of  one  unit  of  inhibition  was  chosen 
to  be  1.25  times  the  strength  of  one  unit  of  excitation 
(1*1. 25E) . This  was  done  by  controlling  the  size  of  the 
IPSfi.  Note  that  neuron  V has  the  highest  intensity  input, 
and  is  therefore  the  retinal  image  of  the  line's  true 
location.  But  due  to  scattering,  all  channels  have  inputs  to 
a lesser  extent.  The  outputs  of  Fig  37c  demonstrate  that 
remarkable  sharpening  has  taken  place  in  that  only  neuron  7 
has  outputs.  The  reason  this  happens  is  that  neuron  V 
produces  the  first  output  and  therefore  inhibits  all  other 
neurons  fees  producing  outputs.  This  and  the  higher  input 
rate  allow  neuron  V to  maintain  dominance. 

Eecause  of  the  complete  nature  of  this  example 
of  line  sharpening,  it  was  decided  to  lower  the  IPSB  to  gain 
some  insight  into  how  much  inhibition  is  needed  tc  give 
satisfactory  sharpening.  Figure  37b  is  the  resulting  output 
for  absolutely  no  inhibition,  and  does  not  appear  to 
represent  any  sharpening.  A better  system  for  comparing 
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sharpness  is  needed,  however.  The  pattern  can  be  represented 
by  a series  of  eight  nuabers  with  the  highest  number 
noraalized  to  100.  The  other  seven  nuabers  would  then  be 
percentages  cf  the  brightest  line.  The  input  distribution  is 
then  represented  as  31,47,78,100,78,47,31,31  (zero 
sharpening)  while  the  output  for  1=1. 25E  would  be 
0,0,0,100,0,0,0,0  (perfect  sharpening) . The  normalized 
intensity  distributions  are  indicated  below  each  plot.  Now 
it  can  be  seen  that  there  is  some  sharpening  even  with  no 
inhibition  at  all.  This  is  termed  sharpening  by  threshold, 
and  will  be  discussed  later. 

The  point  of  this  is  to  note  that  fairly  useful 
sharpening  results  all  the  way  down  to  I=0.156E.  This 
surprizingly  saall  amount  of  inhibition  has  a marked  effect 
because  it  is  actually  enhanced  by  duplication.  That  is,  as 
neurons  0,  V,  and  H generate  strong  outputs,  each  of  then 
contributes  to  the  inhibition  of  neuron  X,  thus  reducing  its 
noraalized  output  count  to  13  vice  44  for  the  no-inhibition 
case. 


Another  question  that  needs  to  be  raised  is  that 
of  how  auch  sharpening  is  necessary  to  an  organism. 
Certainly  it  cannot  benefit  fron  perceiving  a world  in  which 
all  contours  have  been  converted  into  sharp  lines.  The 
answer  to  this  guestion  is  unfortunately  not  available  from 
these  aodels.  But  perceptual  studies  generally  do  not  seen 
to  support  sharpening  as  extreme  as  just  demonstrated.  Does 
this  tend  to  show  that  perhaps  inhibition,  in  real-life 
lateral  inhibition  networks,  is  low  in  magnitude  relative  to 
excitation?  The  answer  is  no,  since  the  artificial  character 
of  the  inputs  just  presented  is  responsible  for  the  high 
degree  of  effectiveness  of  the  network.  In  Fig  37a,  the 
fact  that  all  channels  have  their  initial  input  iaaediately 
(fixed  phase)  is  no  coincidence.  This  was  done  ,to 
deaonstrate  the  input  style  which  produces  the  aost  striking 


example  of  line  sharpening.  Other  input  styles  resulted  in  a 
lesser  degree  of  sharpening  in  some  cases  and  in  bizarre 
effects  in  other  cases.  Specifically/  two  other  input  styles 
were  investigated:  random,  and  constant  frequency  with 
random  phase. 

The  results  of  line  sharpening  runs  with  random 
inputs  are  shown  in  Pigs  40,  41,  and  42.  The  number  of 
inputs  to  each  neuron  was  carefully  chosen  to  be  identical 
to  the  number  for  the  constant  frequency,  fixed  phase  case. 
(This  cannot  be  verified  by  manually  counting  markers  in  Fig 
40a,  since  markers  sometimes  fall  so  close  together  so  as  to 
be  unresolvatle  graphically.  The  counts  were  performed  by 
computer  and  were  verified  to  be  identical  to  those  of  Fig 
37a.)  The  striking  difference  between  random  inputs  and 
constant  frequency,  fixed  phase  inputs  is  that  much  higher 
levels  of  inhibition  are  necessary  if  the  random  line 
sharpening  is  to  even  approach  the  latter.  The  reason  is 
obvious:  the  constant  frequency,  fixed  phase  situation  had  a 
100  percent  chance  that  the  correct  channel  would  fire  and 
thus  inhibit  first,  and  therefore  maintain  dominance.  With 
random  inputs,  bunching  can  occur  and  force  an  output  even 
in  a strongly  inhibited  channel  with  low  average  input 
intensity. 

Examining  the  normalized  output  distributions, 
it  is  clear  that  useful  sharpening  does  not  occur  below 
1*1. 25E  for  this  style  of  random  inputs,  and  the  performance 
of  the  lateral  inhibition  network  can  be  called  marginal. 
The  question  arises  as  to  what  time  sequence  best  represents 
what  the  actual  inputs  to  a real-life  lateral  inhibition 
network  might  be.  Considering  that  the  stimulus  is  a bright 
line  switched  on  at  shortly  before  time  zero,  the  fixed 
phase  input  is  not  totally  unreasonable,  since  all  receptors 
would  begin  to  be  stimulated  at  about  the  same  time,  and 
would  see  a steady  level  of  stimulus  and  hence  fire  at  an 
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unchanging  rate.  But  there  is  certainly  soae  degree  of 
randomness  associated  with  the  process  of  quanta  absorption, 
receptor  potential  generation,  and  spike  production. 
Additionally,  saccadic  eye  movements  would  cause  variations 
in  the  illumination  a particualr  receptor  would  get. 


b.  Extreme  Phase  Sensitivity 

Continuing  to  pursue  the  best  input 
representation,  consider  now  a subject  staring  at  a blank 
wall  (Figs  43  and  44) . If  luminance  were  switched  on  at  just 
before  time  zero,  then  randomness  in  absorption  of  guanta 
and  development  of  the  initial  input  is  represented  by  the 
random  location  of  the  initial  markers  (Fig  44a) . The  fact 
that  the  subject  is  staring  at  a large  wall  of  constant 
luminosity  accounts  for  the  fact  that  each  channel  gets  the 
sane  input  intensity.  Saccadic  eye  movements  would  produce 
no  effects  in  this  example.  Consider  that  once  the  first 
input  is  created,  subsequent  inputs  are  regularly  spaced. 
The  resulting  outputs  are  bizarre,  and  demonstrate  an 
extreme  phase  sensitivity.  Basically,  the  first  channel  to 
generate  an  output  in  such  a situation  will  maintain 
dominance  in  its  locality.  It  is  unlikely  that  this  extreme 
phase  sensitivity  exists  in  nature. 

c.  line  Shift 

If  applied  to  the  diffuse  line  display  with 
random  phase,  this  phenomenon  can  result  in  a channel  with 
lower  input  intensity  gaining  and  keeping  dominance  over  a 
channel  with  higher  input  intensity  (Fig  45) . Note  that 
channel  Y maintains  dominance  even  though  it  is  lower  in 
input  intensity  than  channels  U,  V,  W,  and  X.  The 
competition  between  channels  0 and  Y is  of  interest.  Channel 
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U has  the  first  output,  but  its  inhibition  of  channel  Y 
occurs  during  that  neuron's  refractory  period,  and  is 
therefore  ineffective.  The  inhibition  on  neuron  0 by  the 
first  output  from  neuron  I falls  after  the  refractory  period 
and  is  therefore  effective.  This  effect  indicates  that  the 
delay  of  spread  of  inhibition  is  important  in  spatial  as 
well  as  temporal  phenomena. 

line  shift  is  undesirable  for  an  animal  trying 
to  catch  his  dinner  or  avoid  predators,  and  has  undoubtedly 
been  eliminated  or  minimized  by  randomization  of  inputs, 
optimization  of  spatial  extent  of  inhibitory  spread,  or 
both. 


d.  Optimum  Stimulus  Representation 

The  payoff  to  this  lengthy  discussion  of  hew  to 
represent  the  inputs  is  the  hypothesis  that  a new  genre  of 
random  inputs  needs  to  be  devised.  The  random  inputs  of  Fig 
40a,  generated  by  an  algorithm  described  in  Ref  5,  have  a 
Poisson  distribution  and  seem  to  have  too  much  bunching  for 
the  case  of  a subject  fixing  on  a static  visual  field.  The 
Poisson  input  distribution  could  be  modified  by  a simple 
additional  program  which  would  make  the  input  spacing 
somewhat  more  regular.  Such  a program  would  step  through  the 
poisson  inputs  and  remove  any  marker  found  within  a certain 
small  time  following  any  other  marker.  Additionally,  the 
program  would  insert  a marker  if  a longer  increment  of  time 
were  to  elapse  without  a marker.  The  short  and  long  time 
increments  wculd  be  callable  parameters. 

e.  line  Sharpening  by  Threshold 

There  is  another  conceivable  mechanism  by  which 
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FIGURE  43  “ ABNORMAL  PHASE  SENSITIVITY,  PART  1 
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line  sharpening  could  occur.  By  adjusting  the  threshold  for 
spike  production,  the  nost  intense  channel  can  be  aade  to 
fire  sore  often  in  proportion  to  its  inputs  than  the  less 
intense  channels,  even  in  total  absence  of  lateral 
inhibition.  Such  a nechanisa  aay  very  well  be  in  operation 
in  real  organises,  but  it  is  less  effective  than  the 
powerful  lateral  inhibition,  and  also  considerably  less 
flexible.  That  is,  there  are  nany  parameters  in  lateral 
inhibition  which  are  highly  variable  in  the  acdeling  program 
as  well  as  in  the  organise,  and  all  involve,  in  one  way  or 
another,  different  schenes  of  neural  interconnections. 
Because  of  the  iaaense  number  of  neurons  available,  and  the 
staggering  number  of  connections  a single  neuron  can  sake, 
paraaeters  which  vary  because  of  differences  in  neural 
interconnections  are  realistically  capable  of  wide  latitude. 
It  is  iaprobable  that  spike  threshold  variations  would  be 
capable  of  producing  such  a wide  range  of  effects. 

Threshold  sharpening  can  be  discerned  in  Fig  37. 
Hotice  that  the  nornalized  distribution  goes  froa 
31,47,78,100,78,47,31,31  to  19,44,75,100,75,44,19,19  when 
inhibition  is  reao'  id.  This  is  strictly  a threshold  effect. 
In  Fig  40,  however,  the  distribution  goes  to 
23,38^77,100,85,54,23,31,  which  can  hardly  be  considered 
sharpened.  It  appears  that  the  bunching  of  inputs  in  the 
randoa  input  has  decreased  the  effect  of  threshold 
sharpening. 

4.  Spatial  frequency  Besnonse 


Bith  cnly  eight  channels  available  for  inputting 
spatial  distributions,  any  aeaningful  verification  of  the 
aodulation  transfer  function  (page  26)  is  nearly  iapossible. 


Two  effects  relating  closely  to  spatial  frequency  response 
will,  however,  be  deaonstrated.  These  are,  first,  the 


suppression  of  steady,  uniform  levels  of  illusination 
(representing  a low  spatial  frequency)  and  second,  the 
enhancesent  cf  contrast  at  a border  between  two  regions  of 
uniform  intensity  (Hach  bands) . 

a.  Suppression  of  Low  Spatial  Frequencies 


The  enormous  dynamic  range  of  the  visual  system 
for  light  intensity  is  the  result  of  several  factors.  First, 
there  is  the  fairly  minor  effect  of  change  in  pupillary 
aperture  size,  which  occurs  in  a tine  frame  of  several 
minutes  following  a step  change  in  light  intensity.  Second, 
there  is  the  major  effect  of  rhodopsin  concentration 
changes,  which  occur  over  a thirty  minute  time  period. 
Third,  there  is  the  logarithmic  response  of  receptor  cells. 
Last,  there  is  the  effect  that  an  inhibitory  neural  network 
should  cause  an  enhancement  of  dynamic  range  by  virtue  of 
its  low  response  to  low  spatial  frequencies  [Befs  6 and  17]. 
That  is,  as  few  inputs  arrive,  few  outputs  occur,  and  few 
inhibitions  result,  is  more  inputs  arrive,  outputs  increase, 
but  so  do  inhibitions,  thus  holding  down  the  output  level, 
is  in  regularization  by  a lateral  inhibition  network,  the 
duration  of  the  IPSR  seems  to  be  a key  here,  since  for  this 
amount  of  time,  outputs  are  unlikely,  is  inputs  increase  in 
number,  they  are  more  likely  to  meet  with  a large 
hyperpolarization. 


LI1HXB  allows  quantitative  exploration  of  this 
effect  (Figs  46  and  47).  Kith  random  inputs  totaling  83 
(dim  light),  the  inhibitory  neural  network  produces  15  total 
outputs.  If  the  total  number  of  inputs  is  increased  to  235, 
a factor  of  2.83  increase,  a total  of  41  outputs  are 
produced,  a factor  of  2.73  increase.  This  result  was  quite 
disappointing,  as  the  latter  nuaber  should  have  been  much 
smaller  in  order  to  account  for  any  noticeable  effect  in 
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visual  perception. 


While  this  dynamic  range  enhancement  may  appear 
to  be  only  slight  (or  indeed,  questionable),  aany  factors 
aust  be  considered.  First,  this  effect  is  very  sensitive  to 
paraneter  selection  (as  are  aost  effects  in  neural 
modeling) . The  process  of  natural  selection/evolution  over 
Billions  of  years  would  probably  have  produced  a more 
effective  coabination  of  parameters  in  the  visual  system 
than  can  be  happened  upon  in  the  laboratory  during  modeling. 
Second,  with  only  eight  neurons  in  the  array,  we  are  not 
dealing  with  a true  zero  spatial  frequency,  but  rather  with 
a bright  band  on  an  occult  background.  Third,  the  IINH1B 
model  provides  for  lateral  inhibition  to  occur  whenever 
there  is  an  cutput.  There  is  evidence  that  in  liaulus,  a 
threshold  for  inhibition  exists.  That  is,  if  outputs  are  at 
a low  temporal  freguency,  no  inhibition  would  occur,  but 
above  a certain  threshold,  inhibition  would  begin  to  occur. 
This  would  cause  an  increase  in  the  dynamic  range  by 
increasing  the  output  for  dim  inputs. 

Thus  the  attenuation  of  low  spatial  frequencies 
by  a lateral  inhibition  network  has  not  been  proven  by  this 
example.  To  the  contrary,  the  2.83  fold  increase  of  input 
intensity  producing  a 2.73  fold  increase  in  outputs  casts 
doubt  on  the  suppression  of  low  spatial  frequencies  at  all. 
in  fact,  even  a single  EIPSP  neuron,  when  presented  with 
successively  more  intense  excitatory  inputs,  will  not 
produce  outputs  which  increase  by  the  same  ratio  [Bef  5]. 
This  is  because  as  the  number  of  random  inputs  are 
increased,  any  of  them  which  occur  very  shortly  following  an 
output  are  "wasted",  since  inputs  occurring  during  the 
refractory  period  have  no  effect  on  the  PSP.  Thus  the  very 
slight  observed  suppression  could  have  resulted  from  this 
effect,  and  not  from  and  characteristics  of  the  lateral 
inhibition  network.  It  became  apparent  that  there  are  two 
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sides  to  the  EIPSP  neuron's  output/input  ratio  story.  If  the 
nuaber  of  inputs  is  initially  very  low  (such  as  six  per 
channel) , the  ratio  will  be  lov,  since  inputs  tend  to  occur 
alone,  is  the  number  increases  from  this  very  lov  value,  the 
output/input  ratio  increases  as  inputs  occur  closer 
together,  temporal  summation  cooing  into  play.  A 4.13  fold 
input  increase  producing  a 3.13  fold  output  increase  is  nov 
seen  to  be  such  more  impressive,  since  it  represents  a large 
aaount  of  hidden  suppression  in  overcoming  tne  temporal 
summation  effect  in  the  neuron  output/input  characteristic. 

Bore  runs  were  made  to  attempt  to  prove  or 
disprove  the  existence  of  lov  spatial  frequency  attenuation. 
A new  pair  of  input  fields  was  used,  this  time  temporally 
randoa  but  spatially  very  uniform  at  six  inputs  per  channel 
for  the  dia  and  31  inputs  per  channel  for  the  bright  field 
(a  factor  of  4.13) . After  many  hours  of  parameter  variation, 
the  best  performance  attainable  was  a 3.13  ratio  of  outputs, 
which  is  still  not  overly  impressive.  It  was  decided  to 
determine  what  possible  offsetting  effect  due  to  the  EIPSP 
neuron  input/cutput  characteristic  might  be  present,  so  the 
same  runs  were  made  without  inhibition.  The  result  was 
surprising  in  that  a 4.13  fold  increase  in  inputs  produced  a 
5.S4  fold  increase  in  outputs  without  inhibition.  This  makes 
the  low  freguency  attenuation  of  the  lateral  inhibition 
network  more  impressive. 

One  possible  source  of  poor  perforaance  in 
attenuaticn  of  low  spatial  frequencies  aight  be  that  a 
unifora  field  over  an  array  of  only  eight  neurons  is  too 
snail  to  be  considered  a low  spatial  frequency.  In  order  to 
attempt  to  eliainate  this  effect,  LID=1 1 1 1 1 1 10 1 1 1 1 1 1 1 was 
used.  As  will  be  aentioned  in  the  next  section,  this  LID 
tends  to  give  the  effect  of  an  infinite  array.  The  result 
was  disappointing:  a 4.13  fold  increase  in  input  intensities 
produced  a 3.6  fold  increase  in  output  intensities,  which 
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was  not  as  good  as  other  runs  with  raap  type  LID's. 

This  investigation  should  be  continued  following 
the  development  of  the  improved  random  inputs  discussed 
earlier.  It  is  also  probable  that  the  use  of  several  LINHIB 
processing  stages  in  series  could  improve  the  suppression  of 
lew  spatial  frequencies. 

Perhaps  the  most  impressive  quality  of  the 

retina's  lateral  inhibition  network  is  that  it  allows 

viewing  of  a large  bright  region  and  its  low  contrast 

details  in  one  area  of  the  visual  field  simultaneous  with  a 
large  dim  region  and  its  low  contrast  details.  The  retina, 
with  its  ability  to  adjust  its  overall  sensitivity,  is  the 
living  analogy  to  a photographic  film  which  has  an 
automatically  adapting  speed.  Therefore,  because  of  the 

ability  cf  the  retina  to  have  different  sensitivities  at 
different  regions,  a truer  analogy  would  be  a photographic 
film  which  would  adjust  its  sensitivity  locally  to  enhance 
contrast  at  every  point  in  the  field  of  view  [Ref  17]. 

b.  Bach  Bands 

Rhen  a bright  region  borders  on  a dim  one  with  a 
relatively  sharp  contour  separating  the  two,  the  visual 
system's  spatial  frequency  response  sharpens  the  contrast  at 
the  border.  This  appears  to  the  observer  as  a darkening  of 
v the  dark  region  near  the  border,  and  a lightening  of  the 
light  region  near  the  border,  the  so-called  Mach  bands. 
Consider  the  input  field  of  pig  48,  which  represents  a 
bright  bar  on  a very  dark  background,  'he  outputs  do  not 
have  the  same  distribution  as  the  inputs,  and  represent  a 
crude  Mach  band  phenomenon.  Specifically,  the  input 

distribution  is  flat  while  the  output  is 
83,42,42,56,33,75,8,100.  Recalling  that  the  eight  neuron 
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array  really  represents  eight  neurons  receiving  soae  degree 
of  illuaination  while  all  bordering  neurons  are  in  total 
darkness,  it  is  clear  that  the  edges  of  the  distribution 
(neurons  S and  Z)  should  appear  lighter  if  the  phenoaenon  is 
to  be  present.  They  do  indeed  appear  lighter. 

The  LID  was  chosen  as  000000101000000  for  this 
deaonstraticn  in  order  to  enhance  the  effectiveness.  In 
order  to  understand  this,  consider  why  Mach  bands  appear. 
Central  nenrons  (T,  0,  V,  w , x,  and  7)  receive  double 
inhibition  (cnce  froa  each  iaaediate  neighbor)  whereas  edge 
neurons  (S  and  Z)  receive  only  single  inhibition  (none  froa 
above  S or  below  Z) . By  choosing  LID=1 11111101 1111 11  the 
phenoaenon  can  be  aade  to  vanish,  since  each  neuron  is 
inhibited  by  each  of  the  other  seven  (Pig  49) . Specifically, 
the  output  distribution  is  83,100,100,100,63,100,83,100, 
which  is  nearly  flat,  and  certainly  shew  no  prominent  Mach 
bands.  For  this  saall  network,  this  LID  represents  infinite 
spread  of  inhibition,  and  tends,  to  aodel  an  infinite  array 
of  neurons  (so  long  as  the  distribution  is  uniform) . 

Soae  thought  suggests  that  the  aost  effective 
contour  sharpening  will  result  if  the  diaension  of  spread  of 
inhibition  is  on  the  sane  order  as  the  diaension  of  the 
contour. 


The  concept  of  target  neurons  which  contain  PSP's  which 
vary  fron  a period  of  hyperpolarization  to  a period  of 
relative  depolarization  is  hinted  at  by  Refs  2 and  4.  This 
concept  has  keen  used  in  this  thesis  to  aodel  a speed  band 
pass  gate  as  well  as  to  regularize  a randoa  input  spike 
train.  The  neural  oscillator,  used  to  drive  the  second 
neuron  to  generate  a sinusoidal  PSP,  is  a very  siaple 
circuit,  and  should  therefore  be  sought  in  living  organises. 
The  location  of  a neural  oscillator  would  probably  be 
central,  since  this  is  where  the  target  neurcn  for  the  band 
pass  speed  detector  is  located.  Both  of  these  models  work 
very  well,  and  should  be  considered  as  good  possibilities  as 
to  how  such  phenomena  sight  actually  occur  in  nature. 

The  nodels  presented  herein  for  slow  and  fast  speed 
detectors  utilize  the  concept  of  a lateral  inhibitory 
network  perforaing  temporal  functions  rather  than  the  usual 
spatial  functions.  By  varying  the  1PSR  duration,  extent  of 
inhibitory  spread,  and  delay  in  spread  of  inhibition,  an 
effective  neural  speed  detection  systea  eaerges.  The 
location  cf  such  a circuit  should  be  addressed,  since  in  the 
aodel,  ganglion  outputs  are  sunaed  on  a geniculate  cell, 
whereas  in  the  rabbit,  a single  ganglion  cell  is  the  target 
neuron  (Bef  1 ].  But  it  is  probably  also  true  that  the  rabbit 
has  a later  stage  of  processing  wherein  a geniculate  cell 
serves  as  a target  for  speed  detection.  Rith  this  slight 
reservation,  the  aodel  is  believable  as  well  as  effective. 

The  scheae  used  in  this  thesis  for  establishing  null 
versus  preferred  direction  in  notion  detection  is  less 
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desirable  than  that  of  Fig  24  since  notion  in  the  null 
direction  does  not  produce  a true  null.  This  is  probably  not 
a serious  drawback.  One  aethod  of  ispfoving  the  null  light 
be  to  place  another  processing  stage  in  series. 

The  best  notion  detection  scheae  in  terns  of  sawing 

I 

precious  retinal  space  would  be  to  have  the  retinal  lateral 
inhibition  network  function  siaply  in  a spatial  role 
(sharpening,  contrast  enhancing,  and  dynanic  range 
increasing) . The  gaglion  cell  outputs  would  then  fan  out  to 
subsequent  central  lateral  inhibition  networks  where  null 
versus  preferred  direction  as  well  as  high  pass  or  low  pass 
characteristics  would  be  developed. 

The  lengthy  discussion  of  input  stiaulus  representation 
for  lateral  inhibition  networks  would  apply  centrally,  but 
would  probably  not  be  necessary  retinally,  where 
predominantly  slow  potentials  occur.  Hith  slow  potentials, 
the  probleas  of  regularity  and  phase  would  not  occur.  But  if 
ganglion  cell  outputs  were  fed  to  central  lateral  inhibition 
networks,  such  considerations  would  apply. 

Although  handicapped  by  the  basic  difference  in  fora  of 
input  stiaulus,  the  LIHHIB  program  did  a good  job  in 
exploring  the  effects  of  line  sharpening,  inhibition  and 
disinhibition,  and  Bach  band  fornation.  It  aust  be 
eaphasized,  however,  that  LINHIB  aodels  a central  lateral 
inhibition  network,  whereas  aost  of  these  spatial  phenoaena 
probably  occur  retinally.  The  aodeling  of  spatial  phenoaena 
should,  therefore,  be  taken  less  seriously  than  the  aodeling 
of  temporal  phenoaena. 

The  perforaance  of  LINHIB  in  suppressing  low  spatial 
frequencies  was  not  as  draaatic  as  expected  until 
perforaance  without  inhibition  was  explored.  Then,  it  becaae 
obvious  that  auch  of  the  low  spatial  frequency  suppression 
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was  being  Basked  by  a change  in  the  basic  EIPSP  neuron* s 
output/input  ratio  as  input  density  increases.  To  belabor 
the  point,  the  sane  change  in  output/input  ratio  would  in 
all  probability  not  occur  with  slow  potential  interaction, 
and  the  low  spatial  freguency  suppression  would  be  sore 
narked.  This  represents  additional  evidence  that  low  spatial 
frequencies  are  suppressed  retinally. 


VI.  !£££j}fl£MAm&  ZQZ  ZUXUZZ  ZQRZZiM  &Q&£ 


Hany  physiological  systess  are  ripe  for  cosputer 
■odeling.  One  exasple  is  the  auscle  stretch  receptor  unit. 
Consents  here#  however#  will  be  confined  to  expanding  and 
refining  the  aodeling  of  visual  neurophysiology#  and  the 
neuron  in  general. 

At  present#  all  aodeling  assuaes  that  inforaation  passes 
frca  neurcn  to  neuron  via  the  cheaicai  synapse#  and  that  all 
inforaation  is  in  the  fora  of  action  potentials  or  spikes. 
There  is  a growing  body  of  evidence  that  other  for as  of 
inter-neuxonal  coaaunications  exist.  It  is  a long 
established  fact  that  sensory  receptors  do  not  generate 
spikes#  but  rather  aaintain  a greater  or  lesser  aeabrane 
potential  in  response  to  external  stiauli.  Siailarly# 
horizontal  and  bipolar  cells  in  the  retina  do  not  generally 
exhibit  spike  activity#  but  coaaunicate  with  other  neurons 
by  soae  other  aeans.  Becent  evidence  indicates  that  the 
varying  aeabrane  potential  in  these  cases  nodulates  the 
release  of  neurotransaitter  at  cheaicai  synapses  £Bef  15 j. 
Although  this  neurotransaitter  has  not  yet  been  specified 
for  retinal  neurons  and  receptors#  the  synaptic  vessicles 
can  clearly  be  seen  in  electron  nicrographs.  it  appears  to 
be  a valid  general  principle  that  spikes  are  not  necessary 
for  inter-neuronal  coaaunication#  but  rather  that  their 
purpose  is  to  allow  a suaaed  signal  at  the  axon  hillock  to 
be  coasuaicated  intra-neuronally  to  the  presynaptic  bulbs 
via  a lengthy  axon.  Electrotonic  spread  would  be  inadequate 
because  of  the  losses  which  would  occur  over  the  length  of 
the  axon.  In  order  to  be  as  factual  as  possible#  then#  the 
aodeler  should  undertake  this  sort  of  inter-neuronal 
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coMunicati.cn 


The  fact  that  loss  is  inherent  in  electrotonic  spread  of 
the  PSB  raises  an  interesting  issue.  lamely,  how  such 
■enhrane  potential  change  at  the  synapse  is  necessary  to 
cause  eodulation  of  neurotranseitter  release?  With  spike 
action  potentials,  arrival  is  accoepanied  by  the  release  of 
one  "unitn  of  neurotransaitter.  with  slow  potentials, 
however,  the  length  of  the  axon- like  process  would  affect 
the  change  in  potential  at  the  synapse,  and  therefore  would 
also  affect  the  degree  of  sodulation  of  neurotransaitter 
release.  It  is  not  entirely  reasonable  that  process  length 
should  be  so  important,  but  it  would  serve  as  a mechanise 
whereby  the  strength  of  inhibition  would  decrease  with 
distance  from  the  neuron  initiating  the  inhibition.  Perhaps 
there  is  sose  threshold  for  membrane  potential  change,  and 
so  long  as  the  change  were  greater  than  threshold,  a "unit" 
of  modulation  would  occur.  It  is  doubtful  that  this  point 
could  be  clarified  by  the  type  of  modeling  described  in  this 
thesis. 


In  addition  to  non-spike  inter-neuronal  communication  at 
chenical  synapses,  it  has  been  maintained  for  many  years 
that  electrical  synapses  exist.  The  najor  evidence  seems  to 
be  twofold.  First,  specialized  regions  at  which  membranes 
come  into  ultra  close  proximity  (on  the  order  of  20 
angstroms)  have  been  observed,  and  are  believed  to  be 
electrical  synapses.  Second,  aicroelectrode  studies  in  many 
instances  have  produced  neuron  pairs  whose  PSP's  vary  nearly 
synchronously,  but  not  identically,  £Bef  14]  in  response  to 
induced  menbrane  currents  in  only  one  of  the  neurons.  The 
principle  drawback  to  quick  acceptance  of  this  scheme  is 
that  the  mechanism  of  coupling  has  not  been  elucidated.  If  a 


current  at  the  electrical  synapse  is  responsible  for  the 
change  in  nenbrane  potential  of  the  postsynaptic  neuron, 
then  the  unquestioned  existence  of  nenbrane  capacitance 


(which  would  require  an  integration  tine  in  order  for 
synaptic  current  to  produce  a postsynaptic  voltage  change) 
would  sake  simultaneous  variation  of  aeabrane  potentials 
iapossible.  If  current  at  such  a specialized  site  causes 
soae  postsynaptic  aeabrane  permeability  change  which  would 
then  spread,  causing  a rapid  change  in  PSP,  this  would  be  a 
aore  plausable  explanation.  No  such  aechanisa  has  been 
found,  however.  Because  electrotonic  coupling  seeas  to  be 
here  to  stay,  the  aodeler  should  include  it  in  his 
repertoire. 

One  iaproveaent  in  the  present  way  the  PSP  is  computed 
aight  be  in  order.  Currently,  a large  nuaber  of  closely 
spaced  I-inputs  will  drive  the  PSP  negative  until  the 
largest  negative  integer  capable  of  being  represented  by  one 
coaputer  word  is  reached,  in  a real  neuron,  however,  there 
is  a liait  to  the  aaount  of  hyperpolarization  which  can 
occur.  It  is  the  ion  concentrations,  both  inside  and  outside 
the  neuron,  as  well  as  the  aeabrane' s peraeability  for  each 
ion  which,  when  plugged  into  the  Nernst  eguation  [Be f 9], 
give  seabrane  potential.  Excitatory  as  well  as  inhibitory 
inputs  alter  aeabrane  potential  by  altering  the  aeabrane' s 
peraeability  for  sodiua  and/or  potassiua  ions.  Thus,  there 
is  a liait  to  the  aaount  of  hyperpolarization  which  can 
occur;  this  liait  should  be  incorporated  into  the  aodels. 

There  is  one  iaproveaent  which  could  be  aade  in  the 
fatigue/facilitation  prograa  (PSPFAT) . At  present,  the 
running  average  is  calculated  for  each  word  of  the  IK 
processing  block.  That  nuaber  is  then  used  to  adjust  the 
sagnitude  of  the  PSB  for  all  inputs  since  the  end  of  the 
last  refractory  period.  Thus,  the  shape  of  the  PSB  for  a 
given  input  would  vary  froa  the  accepted  shape  because  of 
its  being  weighted  differently  at  different  processing 
words.  A sore  realistic  aethod,  but  one  which  involves  aore 
processing,  would  be  to  assign  a weight  to  the  PSB  of  a 


given  inpat  at  the  tiae  the  input  occurs.  Then,  whenever 
that  particular  PSH  were  sunned  to  forn  a PSP  word,  the 
original  weight  would  be  used.  This  could  be  handled  by 
using  the  high  bytes  of  the  words  of  the  age  factor  stack  as 
the  weight.  This  nethod  would  be  nore  consistent  with  the 
nechanisn  of  fatigue  in  which  the  anount  of  neurotransnitter 
released  changes. 

As  nentioned  in  the  section  on  notion  detection,  the 
null  versus  preferred  detector  using  LINHIB  differs  fron 
that  proposed  by  Ref  12  and  pictured  in  Pig  24.  This  neural 
circuit  would  be  very  worthwhile  to  nodel. 

Discussed  in  the  section  on  oscillatory  PSP's  was  the 
problen  of  phase  in  a speed  band  pass  gate.  The  thought  that 
perhaps  the  oscillator  neuron  is  strobed  by  a noving  target 
of  any  speed  such  that  the  proper  phase  is  produced  in  the 
oscillatory  PSP  is  an  intriguing  one  well  worth  an  attenpt 
at  nodel ing. 

V 

The  ncving  target  nodels  using  LIHHIS  all  have  one 
difficulty  which  should  eventually  be  corrected.  For  a 
ncving  target,  the  nodels  work  satisfactorily.  But  consider 
the  response  to  a bright  stinulus  which  sinultaneously 
illuninates  all  eight  neurons.  All  would  have  outputs,  and 
the  speed  gate  target  neuron  would  have  a false  sense  of  a 
noving  target  at  the  pass  speed.  The  solution  to  this  is  to 
introduce  fatigue  into  each  of  the  eight  neurons.  In  this 
way,  significant  firing  would  occur  in  all  eight  only 
initially,  and  the  target  neuron  would  have  output  only  at 
first. 

Another  inproveaent  to  LIMHIB  would  involve  the 
directionality  of  the  raap  delay.  At  present,  either 
constant  delay  in  both  directions  or  rasp  delay  in  both 
directions  is  selectable.  If  one  could  select  constant  delay 
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in  one  direction  and  raop  delay  in  the  other,  then  a neuron 
with  null  and  preferred  directions  which  could  have  high 
speed  pass  properties  in  the  preferred  direction  could  be 
aodeled.  Sixilarly,  if  different  PSB  durations  could  be 
specified  for  different  directions  in  LINHIB,  then  a network 
which  could  have  a null  direction,  and  detect  only  slow 
targets  in  the  preferred  direction  could  be  aodeled. 


With  only  eight  channels  available  in  LINHIB,  the 
investigation  of  spatial  phenomena  is  severely  restricted. 
Any  aeaningful  evaluation  of  the  spatial  frequency  response 
of  such  a network  would  require  far  nore  than  eight  channels 
of  resolution.  If  100  channels  were  available,  one  could 
provide  inputs  which  had  various  spatial  frequency 
characteristics  (such  as  a sinusoidal  or  square  wave 
grating) , and  derive  the  outputs.  The  spectra  for  input  and 
output  could  then  be  found  by  Fourier  transfora  of  a block 
of  100  words  where  each  word  represents  the  count  in  a 
channel,  thus  providing  a transfer  function  in  the  frequency 
doaain.  The  parameters  to  be  varied  would  involve  the 
distance  cf  inhibitory  spread  and  the  spatial  shape  of  the 
fall-off.  Using  the  general  philosophy  of  LINHIE,  but 
requiring  enlargeaent,  rearrangeaent,  and  reorganization, 
one  could  expand  the  nuaber  of  channels  to  a very  large 
nuaber.  This  would  involve  shortening  the  block  length, 
perhaps,  but  would  also  require  the  technique  of  processing 
all  blocks  for  several  words,  storing  results  on  disk, 
processing  the  next  several  words  for  all  blocks,  and  so  on 
until  ccaplete.  Plotting  would  then  require  recall  of  blocks 
froa  the  disk.  This  modeling  project  would  be  an 
undertaking  of  aajor  proportions. 


■eural  circuits  which  recognize  static  patterns  exist 
and  should  be  fairly  easy  to  aodel.  Examples  of  static 
patterns  which  have  been  observed  to  produce  strong  outputs 
frca  cortical  target  cells  are  borders  between  light  and 


dark  regions  and  light  or  dark  slits  or  narrow  bars  on 
contrasting  backgrounds.  Pattern  orientation  is  of  extreme 
importance  in  these  examples  [Ref  10]. 


The  importance  of  pattern  recognition  to  living 
organisms  cannot  be  overstated.  It  is  amazing  that  a person 
can  recognize  a dinner  plate  even  though  it  is  tilted  such 
that  it  appears  elliptical.  Even  more  amazing  is  the  fact 
that  an  oval  serving  platter  also  appears  elliptical  when 
tilted,  but  this  would  seldom  be  confused  with  the 
elliptical  shape  of  the  dinner  plate.  It  is  true  that  this 
sort  of  recognition  would  require  far  more  complicated 
neural  circuitry  than  recognition  of  a slit  or  border. 
Nonetheless,  the  principles  are  probably  the  same.  The 
target  neurons  of  cortical  area  17  (the  primary  visual  area) 
identify  small  simple  patterns.  The  columnar  organization  of 
area  17  allows  cortical  area  18  (the  secondary  visual  area) 
to  piece  together  long  lines,  bars,  borders,  and  shapes  from 
short  patterns.  Cortical  area  19  (the  visual  association 
area)  performs  the  tasks  of  putting  long  lines  and  shapes 
together  into  an  overall  perception,  and  of  bringing  the 
individual's  past  visual  experiences  into  comparison  with 
the  current  scene.  This  is  a giant  step  representing  a 
massive  amount  of  processing,  but  in  all  likelihood,  many 
repetitions  of  a few  simple  circuits  are  responsible  fcr  it. 


The  modeling  of  pattern  recognition  circuits  is  a large 
area,  and  a very  interesting  one,  as  it  relates  so  closely 
to  one's  most  valuable  sense.  Creativity  in  devising  and 
modeling  neural  circuits  capable  of  recognizing  patterns 
could  aid  in  understanding  the  neural  circuits  which  perform 
pattern  recognition  in  real  visual  systems.  The  modeling  of 
pattern  recognition  circuits  is  certainly  one  of  the  most 
worthwhile  areas  for  future  modeling. 
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